Sample Header Ad - 728x90

Limit Openldap Transaction Log Disk Usage

5 votes
1 answer
2613 views
Openldap (specifically version 2.4) stores transaction history in log files by default in the ldap data directory (so /var/lib/ldap/log.###########). Currently these log files take up a lot of space, are never removed automatically, and grow infinitely. Manual removal for old logs works fine, but I'd like to limit the amount of logs slapd keeps automatically. # MY SCENARIO # I know that these transaction logs are used to recover ldap in case of a catastrophic failure. In my scenario ldap is regularly wiped and populated via a script (this isn't used for system login accounts). Because of this I don't need to concern myself with recovery, in case of a failure it's acceptable to run the script again. On the other hand, the regular wipe/population of ldap includes a lot of transactions, so these transaction logs build up pretty quickly. # LOGROTATE # logrotate has potential here, but if the most recent transaction log is ever removed then slapd will fail to start (it will complain about needing to perform recovery). Because I can't rely on the log names (because slapd keeps many small logs, incrementing the log file number as it goes) I'd like to use the Berkeley DB settings which creates these logs. I can count on the access/creation dates (most recent modify date is the most recent transaction log), but I'd still prefer to use Berkeley if possible. # DB_CONFIG # The settings for the transactions logs are said to be controlled by the Berkeley DB settings in /var/lib/ldap/DB_CONFIG. The example DB_CONFIG that comes with openldap specifies some transaction log settings: set_lg_regionmax 262144 set_lg_bsize 2097152 According to the Oracle documentation on Berkeley: set_lg_regionmax: Set the size of the underlying logging area of the Berkeley DB environment, in bytes. The log region is used to store filenames, and so may need to be increased in size if a large number of files will be opened and registered with the specified Berkeley DB environment's log manager. So this seems to just set the size of the file that tracks the transaction log files. set_lg_bsize: Set the size of the in-memory log buffer, in bytes. This seems to control how much RAM is allotted to the transaction buffer. The log.########### files in the ldap data directory are all 10485760 bytes which seems to correspond closely to set_lg_bsize (10485760 / 5 = 2097152 = set_lg_bsize) though I'm not sure if this is a coincidence. My interpretation of this is that $lg_bsize amount of transaction history is stored in memory at a time. When this limit is exceeded it pushes some of the transaction history to the most recent log file, and creates a new log if the current log reaches a certain size. # DB_LOG_AUTOREMOVE # According to the Berkeley documentation transaction logs can be removed by setting the flag DB_LOG_AUTOREMOVE in the DB_CONFIG. DB_LOG_AUTOREMOVE: If set, Berkeley DB will automatically remove log files that are no longer needed. However when I added this to the DB_CONFIG: set_flags DB_LOG_AUTOREMOVE and restarted slapd I didn't notice a difference. I removed the old transaction logs and ran the ldap population script that I have, and was able to rack up 290MB in transaction logs. It still doesn't seem to be limiting the logs at all. The reason for this may be related to the phrase: that are no longer needed # Actual Question # How does one configure the automatic removal of slapd's transaction logs using the Berkeley DB DB_CONFIG file?
Asked by Centimane (4520 rep)
Oct 5, 2016, 05:14 PM
Last activity: Jul 30, 2025, 11:08 AM