Sample Header Ad - 728x90

Replicating/Inserting 300M MYSQL records from 6 Remote Servers on Local using Unique Identifier

1 vote
0 answers
52 views
I have 6 remote servers that dial phone numbers daily. I have a total record count of over 300 million combined and now I want to procress on them. The table data looks like this using this query
SELECT lead_id,
    entry_date,
    modify_date,
    status,
    list_id,
    phone_number,
    called_count
FROM vicidial_list
WHERE list_id = '202202183'
LIMIT 10;
lead_id  entry_date                   modify_date  status    list_id  phone_number  called_count  
-------  -------------------  -------------------  ------  ---------  ------------  --------------
8455687  2022-02-18 12:08:51  2022-02-18 14:08:17  PU      202202183  XXXXXX1775                 3
8455688  2022-02-18 12:08:51  2022-02-18 13:57:20  PU      202202183  XXXXXX0485                 2
8455689  2022-02-18 12:08:51  2022-02-18 13:57:20  NA      202202183  XXXXXX2277                 2
8455690  2022-02-18 12:08:51  2022-02-18 14:53:55  PU      202202183  XXXXXX8044                13
8455691  2022-02-18 12:08:51  2022-02-18 13:57:20  NA      202202183  XXXXXX1802                 2
8455692  2022-02-18 12:08:51  2022-02-18 13:15:55  NA      202202183  XXXXXX9687                 1
8455693  2022-02-18 12:08:51  2022-02-18 13:15:55  NA      202202183  XXXXXX0603                 1
8455694  2022-02-18 12:08:51  2022-02-18 13:15:55  NA      202202183  XXXXXX1667                 1
8455695  2022-02-18 12:08:51  2022-02-18 14:53:02  AA      202202183  XXXXXX8783                13
8455696  2022-02-18 12:08:51  2022-02-18 13:15:55  NA      202202183  XXXXXX2902                 1
I need to perform queries on these but the problem is: 1. I don't want to manipulate data on 6 Remote Master DB's. Becuase this slows down performance of dialing solution during production hours. 2. I want to be able to perform queries (add columns to default table and update it) on one combined server and not individually on each servers. I want to be replicate/insert them on Local DB where I want to add two more columns in addititon to existing columns where: 1. One new column is "Suffix of DB IP+lead_id" e.g. for server W.X.Y.Z, "Z8455687" as first entry. 2. Another column, which shows count_exported, default is 0. As I have no experience of DB Administration, I am looking for ways to solve this. I want to replicate these on one local system daily during non production hours? Pleae help me how to do this. Do I use MySQL sheduler or cronjob? And how to go about it. Following is how I want my new table to look, if my remote DB IP suffix are, A,B,C,D,E and F:
srv_id   server_  lead_id  entry_date                   modify_date  status    list_id  phone_number  called_count  called_count  
-------  -------  -------  -------------------  -------------------  ------  ---------  ------------  ------------  --------------
X.Y.Z.A  A8455687  8455687  2022-02-18 12:08:51  2022-02-18 14:08:17  PU      202202183  XXXXXX1775               3              0
X.Y.Z.A  A8455688  8455688  2022-02-18 12:08:51  2022-02-18 13:57:20  PU      202202183  XXXXXX0485               2              0
X.Y.Z.B  B8455689  8455689  2022-02-18 12:08:51  2022-02-18 13:57:20  NA      202202183  XXXXXX2277               2              0
X.Y.Z.B  B8455690  8455690  2022-02-18 12:08:51  2022-02-18 14:53:55  PU      202202183  XXXXXX8044              13              0
X.Y.Z.C  C8455691  8455691  2022-02-18 12:08:51  2022-02-18 13:57:20  NA      202202183  XXXXXX1802               2              0
X.Y.Z.D  D8455692  8455692  2022-02-18 12:08:51  2022-02-18 13:15:55  NA      202202183  XXXXXX9687               1              0
X.Y.Z.E  E8455693  8455693  2022-02-18 12:08:51  2022-02-18 13:15:55  NA      202202183  XXXXXX0603               1              0
X.Y.Z.F  F8455694  8455694  2022-02-18 12:08:51  2022-02-18 13:15:55  NA      202202183  XXXXXX1667               1              0
X.Y.Z.F  F8455695  8455695  2022-02-18 12:08:51  2022-02-18 14:53:02  AA      202202183  XXXXXX8783              13              0
X.Y.Z.D  D8455696  8455696  2022-02-18 12:08:51  2022-02-18 13:15:55  NA      202202183  XXXXXX8783               1              0
Asked by Aun Zaidi (111 rep)
Feb 19, 2022, 12:39 AM
Last activity: Feb 20, 2022, 11:21 PM