Sample Header Ad - 728x90

check patterns that don't exist in sqlite

4 votes
3 answers
329 views
I explained a similar situation with plain text files on https://unix.stackexchange.com/questions/29624/grep-huge-number-of-patterns-from-huge-file/30144 . Many people there said I should, so now I'm migrating my data to a sqlite database: I have a file from which I extract about 10,000 patterns. Then I check if the database doesn't contain such patterns. If it doesn't, I need to save them externally in file for further processing: for id in $(grep ^[0-9] keys); do if [[ -z $(sqlite3 db.sqlite "select id from main where id = $id") ]]; then echo $id >>file fi done Since I'm new to SQL, I couldn't find a simple way to do this. Also, this loop is useless as it is 20 times slower than what I achieved with awk on the mentioned URL. Since the database is huge, keeps growing, and I run this loop very frequently, is it possible to make this faster?
Asked by admirabilis (4792 rep)
Jan 31, 2012, 12:54 AM
Last activity: Nov 11, 2023, 01:26 PM