I am using openstack object storage in my production, and faced with big problem - low replication speed.
I have cluster with replication factor 3, with 3 zones and 25 hdds per zone. My container has ~100 million small objects. I've added new hdds (10 per zone) and start rebalance my cluster, and swift report me that rebalancing will be finishing after ~1 year. Swift uses rsync, and I think if I could try copy this objects by hand using rsync - it will be more fast than swift do it.
Is there any way to increase replication speed in openstack swift ? I have a feel that swift makes a "pity" for my HDDs and not using 100% capacity to make replication proccess faster. I tried to ask this in Openstack Forum but there is no answers(.
**This is my /etc/swift/object-server.conf**
[DEFAULT]
bind_port = 6000
user = swift
swift_dir = /etc/swift
devices = /mnt/swift
mount_check = True
log_level = ERROR
conn_timeout = 5
container_update_timeout = 5
node_timeout = 5
max_clients = 4096
[pipeline:main]
pipeline = healthcheck recon object-server
[app:object-server]
use = egg:swift#object
replication_concurrency = 1500
replication_one_per_device = False
replication_lock_timeout = 30
[filter:healthcheck]
use = egg:swift#healthcheck
[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift
recon_lock_path = /var/lock
[object-replicator]
concurrency = 1500
run_pause = 5
interval = 5
log_level = DEBUG
stats_interval = 10
rsync_io_timeout = 60
[object-reconstructor]
[object-updater]
concurrency = 200
interval = 20
slowdown = 0.008
log_level = DEBUG
[object-auditor]
interval = 300
[filter:xprofile]
use = egg:swift#xprofile
**This is /etc/rsyncd.conf**
[object]
path = /mnt/swift
read only = false
write only = no
list = yes
incoming chmod = 0644
outgoing chmod = 0644
max connections = 1500
lock file = /var/lock/object.lock
Asked by Kostyantyn Velychkovsky
(41 rep)
Jan 19, 2018, 12:12 PM
Last activity: May 16, 2023, 06:56 AM
Last activity: May 16, 2023, 06:56 AM