Sample Header Ad - 728x90

How I can improve openstack swift replication speed?

4 votes
0 answers
589 views
I am using openstack object storage in my production, and faced with big problem - low replication speed. I have cluster with replication factor 3, with 3 zones and 25 hdds per zone. My container has ~100 million small objects. I've added new hdds (10 per zone) and start rebalance my cluster, and swift report me that rebalancing will be finishing after ~1 year. Swift uses rsync, and I think if I could try copy this objects by hand using rsync - it will be more fast than swift do it. Is there any way to increase replication speed in openstack swift ? I have a feel that swift makes a "pity" for my HDDs and not using 100% capacity to make replication proccess faster. I tried to ask this in Openstack Forum but there is no answers(. **This is my /etc/swift/object-server.conf** [DEFAULT] bind_port = 6000 user = swift swift_dir = /etc/swift devices = /mnt/swift mount_check = True log_level = ERROR conn_timeout = 5 container_update_timeout = 5 node_timeout = 5 max_clients = 4096 [pipeline:main] pipeline = healthcheck recon object-server [app:object-server] use = egg:swift#object replication_concurrency = 1500 replication_one_per_device = False replication_lock_timeout = 30 [filter:healthcheck] use = egg:swift#healthcheck [filter:recon] use = egg:swift#recon recon_cache_path = /var/cache/swift recon_lock_path = /var/lock [object-replicator] concurrency = 1500 run_pause = 5 interval = 5 log_level = DEBUG stats_interval = 10 rsync_io_timeout = 60 [object-reconstructor] [object-updater] concurrency = 200 interval = 20 slowdown = 0.008 log_level = DEBUG [object-auditor] interval = 300 [filter:xprofile] use = egg:swift#xprofile **This is /etc/rsyncd.conf** [object] path = /mnt/swift read only = false write only = no list = yes incoming chmod = 0644 outgoing chmod = 0644 max connections = 1500 lock file = /var/lock/object.lock
Asked by Kostyantyn Velychkovsky (41 rep)
Jan 19, 2018, 12:12 PM
Last activity: May 16, 2023, 06:56 AM