**Description:** RocksDB is a key-value storage so we can simply serialise the list of objects & store the value corresponding to a key. This would be ok if the data in the list is small enough. But if the list is large and ever increasing in size then we would need the data paginated. So in this case storing the entire serialised list data corresponding to a single key would not be a good idea; as there would be a performance issue since every time a new data is inserted into the list this very large value would need to be read & updated also during read time when showing list to user entire value would be retrieved while only a part of it was needed by the user. Ex: Let’s say we want to store orders placed by user in rocksDB. Then we could store this order data in following way in RockDB “u:1:li:o” : Serialised([O1{}, O2{},….On{}]). But if there are thousands of orders placed by user and we would like to retrieve the orders in form of pages (10 or 20 records at a time). So storing thousands of order in same key and retrieving entire data from that key & then giving required 10-20 records won’t be a good idea. Also adding new order by user to same key will affect the performance as described above. So I am working to design schema for efficiently storing and retrieving such large lists in RocksDB. If you can give your suggestions on schema design that would be great & very much helpful.

I am thinking about making a music database. Something where you can search by artist, album or song. This seems to be a Many-to-many model, as an album can have multiple songs, and the same song can be on different albums. Currently I am just using some JSON files, but this isnt really robust, as it can only represent hierarchical data. This would just be a personal project, so I dont need anything too involved. I have looked at SQLite, but also Document-oriented databases. I like the idea of being able to persist the data as JSON or CSV files. What is a good option for my use case? --- Response to some questions: > Do you want a free solution or need to use an enterprise level system like > Microsoft SQL Server which costs money. Free options only > How far do you expect to scale your database?...in other words, in the > foreseeable future roughly how big do you expect it to get (data-wise), how > many users do you expect will be using it both overall and at a given instance > of time, how often will data change inside of it? Most likely it would be a tool that only has one user at a time. So either I would use it myself, or someone might download my tool and use it for themselves. > What does your current technology stack look like? I would prefer something with of of these if possible: C#, D, Go, Nim or Rust. Although I have seen many options in C and C++, so I might have to end up with one of those. > I don't think any modern solution out there persists the underlying data of > the database in JSON or CSV files (though I could be mistaken). I think Document-oriented database offer this, but I could be misunderstaning it. Also I had seen Export to CSV for SQLite.

When configuring a MySQL or MariaDB server, the terms '**standalone**' and '**embedded**' are referred to, but without reference to what they are, or what they do that is different from each other. ---------- * What does the term 'embedded' refer to? Is this like on an IP camera or something? * Also are these the **only** the two types of MariaDB/MySQL servers? Or are there other similar servers, like for example are 'galera', or enterprise other types of servers? * What is the difference between these two or other server types? In other words, what are they used for that is different, or how are they installed that are different, etc. ---------- Note: When installing, these groups seem to apply: [server] apply to both mysqld and embedded (note: 'mysqld' = 'standalone') [mysqld] apply to only mysqld [embedded] apply to only embedded ---------- *From context: in Debian desktop, I think I'm running a 'standalone' server only.*

For sentimental reasons, I'm trying to export my text messages from an old feature phone (LG Cosmos 2, circa 2011) to a flat file (CSV or the like). I was able to successfully transfer the entire contents of the phone's file system to my computer, but the messages are locked in a database format that I do not recognize. I believe the relevant directories and files are as follows:

/DB
  /SchemaFiles
    /Handset
      inboxSchema.dat
  /SysDB
    SMSInboxDB.vol
    SMSInboxDB.vol.log
    SMSInboxDB.vol.sys

I believe the files in SysDB are the actual database volume files, and cannot be interpreted as plain text. However, the inboxSchema.dat file *can* be read as plain text, and contains the following:

# SMS Inbox Schema 
schema SMSInboxSchema
VERSION 0.8//SMSIN_MSG_CHARLEN

table Inbox maxrecord = 300
{
	U, SYNC_OFF, SMSIN_INBOXINDEX,			1100,	USHORT,		INDEX_KEY_ON,	ON;		# smsInboxIndex => Primary Key
	U, SYNC_OFF, SMSIN_INBOXID,				1101,	ULONG,		INDEX_KEY_OFF,	ON;		# smsInboxId 
	U, SYNC_OFF, SMSIN_MSGID,				1102,	USHORT,		INDEX_KEY_OFF,	ON;		# smsMsgID
	U, SYNC_OFF, SMSIN_CATEGORY,			1103,	BYTE,		INDEX_KEY_OFF,	ON;		# smsCategory
	U, SYNC_OFF, SMSIN_SYSTEMTIME,			1104,	ULONG,		INDEX_KEY_ON,	ON;		# smsSystemTime => Soring Key
	U, SYNC_OFF, SMSIN_MCTIME,				1105,	DATE,		INDEX_KEY_OFF,	ON;		# smsMCTimt
	U, SYNC_OFF, SMSIN_BINMCTIME,			1106,	ULONG,		INDEX_KEY_OFF,	ON;		# smsBinMCTime
	U, SYNC_OFF, SMSIN_READFLAG,			1107,	BYTE,		INDEX_KEY_OFF,	ON;		# smsReadflag
	U, SYNC_OFF, SMSIN_LOCKFLAG,			1108,	BYTE,		INDEX_KEY_OFF,	ON;		# smsLockflag
	U, SYNC_OFF, SMSIN_DUPLICATEFLAG,		1109,	BYTE,		INDEX_KEY_OFF,	ON;		# smsDuplicateflag
	U, SYNC_OFF, SMSIN_PRIORITYFLAG,		1110,	BYTE,		INDEX_KEY_OFF,	ON;		# smsPriorityflag
	U, SYNC_OFF, SMSIN_PRIVACYFLAG,			1111,	BYTE,		INDEX_KEY_OFF,	ON;		# smsPrivacyflag
	U, SYNC_OFF, SMSIN_EMSUDHPRESENT,		1112,	BYTE,		INDEX_KEY_OFF,	ON;		# smssmsEMSUdhPresent
	U, SYNC_OFF, SMSIN_MSGENCODINGTYPE,		1113,	BYTE,		INDEX_KEY_OFF,	ON;		# smsMsgEncodingType
	U, SYNC_OFF, SMSIN_NUMOFSEGMENTS,		1114,	BYTE,		INDEX_KEY_OFF,	ON;		# smsNumOfSegments
	U, SYNC_OFF, SMSIN_TITLEMESSAGE,		1115,	STRING(301),	INDEX_KEY_OFF,	ON;		# smsTitleMessage
	U, SYNC_OFF, SMSIN_NUMOFVOICEMAILS,		1116,	BYTE,		INDEX_KEY_OFF,	ON;		# smsNumOfVoicemails
	U, SYNC_OFF, SMSIN_EMSCTRLDATA,			1117,	BYTE,		INDEX_KEY_OFF,	ON;		# smsEMSCtrlData
	U, SYNC_OFF, SMSIN_EMSCONCATREFRENCE,	1118,	INT,		INDEX_KEY_OFF,	ON;		# smsEMSConcatRefrence
	U, SYNC_OFF, SMSIN_EMSCONCATSTATUS,		1119,	BYTE,		INDEX_KEY_OFF,	ON;		# smsEMSConcatStatus
	U, SYNC_OFF, SMSIN_CBADDRESS,			1120,	STRING(49),	INDEX_KEY_OFF,	ON;		# smsAddress
	U, SYNC_OFF, SMSIN_CBADDRESSLEN,		1121,	BYTE,		INDEX_KEY_OFF,	ON;		# smsAddress_len
	U, SYNC_OFF, SMSIN_FRADDRESS,			1122,	STRING(49),	INDEX_KEY_ON,	ON;		# smsAddress
	U, SYNC_OFF, SMSIN_FRADDRESSLEN,		1123,	BYTE,		INDEX_KEY_OFF,	ON;		# smsAddress_len
	U, SYNC_OFF, SMSIN_FRINTRIMEMAIL,		1124,	BYTE,		INDEX_KEY_OFF,	ON;		# smsIntrimEmail
	U, SYNC_OFF, SMSIN_FRNAME,				1125,	STRING(33),	INDEX_KEY_OFF,	ON;		# smsName
	U, SYNC_OFF, SMSIN_FRPBMATCHICON,		1126,	BYTE,		INDEX_KEY_OFF,	ON;		# smsPBMatchIcon
	U, SYNC_OFF, SMSIN_FRPBID,				1127,	USHORT,		INDEX_KEY_ON,	ON;		# smsPBMatchFilePos
	U, SYNC_OFF, SMSIN_SORTKEY,				1129,	STRING(49),		INDEX_KEY_OFF,	ON;		# for name/number sorting
	U, SYNC_OFF, SMSIN_SORTTIME,			1130,	ULONG,		INDEX_KEY_OFF,	ON;		# for time sorting in desc
}
multikey Inbox
{
	NORMALVALUE: SMSIN_SORTKEY, SMSIN_SYSTEMTIME;
	NORMALVALUE: SMSIN_SORTKEY, SMSIN_SORTTIME;
}

table InboxUserData maxrecord = 6000, maxviewsize = 20
{
	U, SYNC_OFF, SMSIN_INBOXINDEX,			1100,	USHORT,		INDEX_KEY_OFF,	ON;		# smsInboxIndex
	U, SYNC_OFF, SMSIN_SEGINDEX,			1140,	BYTE,		INDEX_KEY_OFF,	ON;		# smsSegIndex
	U, SYNC_OFF, SMSIN_MSG,				1141,	BINARY(220),	INDEX_KEY_OFF,	ON;		# smsMsgLen, smsMsg
	U, SYNC_OFF, SMSIN_MSG_CHARLEN,			1142,	BYTE,		INDEX_KEY_OFF,	ON;		# userdataCharLen(before encoding)
}
multikey InboxUserData
{
	NORMALVALUE:SMSIN_INBOXINDEX, SMSIN_SEGINDEX;
}

table CMAS maxrecord = 100
{
	U, SYNC_OFF, SMSCMAS_INBOXINDEX,		1170,	USHORT,		INDEX_KEY_ON,	ON;		# SMSCMASboxIndex => Primary Key
	U, SYNC_OFF, SMSCMAS_INBOXID,			1171,	ULONG,		INDEX_KEY_OFF,	ON;		# smsInboxId 
	U, SYNC_OFF, SMSCMAS_MSGID,			1172,	USHORT,		INDEX_KEY_OFF,	ON;		# smsMsgID
	U, SYNC_OFF, SMSCMAS_CATEGORY,			1173,	BYTE,		INDEX_KEY_OFF,	ON;		# smsCategory
	U, SYNC_OFF, SMSCMAS_SYSTEMTIME,		1174,	ULONG,		INDEX_KEY_ON,	ON;		# smsSystemTime => Soring Key
	U, SYNC_OFF, SMSCMAS_MCTIME,			1175,	DATE,		INDEX_KEY_OFF,	ON;		# smsMCTimt
	U, SYNC_OFF, SMSCMAS_BINMCTIME,			1176,	ULONG,		INDEX_KEY_OFF,	ON;		# smsBinMCTime
	U, SYNC_OFF, SMSCMAS_READFLAG,			1177,	BYTE,		INDEX_KEY_OFF,	ON;		# smsReadflag
	U, SYNC_OFF, SMSCMAS_LOCKFLAG,			1178,	BYTE,		INDEX_KEY_OFF,	ON;		# smsLockflag
	U, SYNC_OFF, SMSCMAS_DUPLICATEFLAG,		1179,	BYTE,		INDEX_KEY_OFF,	ON;		# smsDuplicateflag
	U, SYNC_OFF, SMSCMAS_PRIORITYFLAG,		1180,	BYTE,		INDEX_KEY_OFF,	ON;		# smsPriorityflag
	U, SYNC_OFF, SMSCMAS_PRIVACYFLAG,		1181,	BYTE,		INDEX_KEY_OFF,	ON;		# smsPrivacyflag
	U, SYNC_OFF, SMSCMAS_MSGENCODINGTYPE,		1182,	BYTE,		INDEX_KEY_OFF,	ON;		# smsMsgEncodingType
	U, SYNC_OFF, SMSCMAS_MSG_CHAR,			1183,	BINARY(220),	INDEX_KEY_OFF,	ON;		# smsCMASMessage
	U, SYNC_OFF, SMSCMAS_MSG_CHAR_LEN,		1184,	BYTE,		INDEX_KEY_OFF,	ON;		# smsCMASMessageLength
	U, SYNC_OFF, SMSCMAS_FRADDRESS,			1185,	STRING(49),	INDEX_KEY_ON,	ON;		# smsAddress
	U, SYNC_OFF, SMSCMAS_FRADDRESSLEN,		1186,	BYTE,		INDEX_KEY_OFF,	ON;		# smsAddress_len
	U, SYNC_OFF, SMSCMAS_SORTKEY,			1187,	STRING(49),	INDEX_KEY_OFF,	ON;		# for name/number sorting
	U, SYNC_OFF, SMSCMAS_SORTTIME,			1188,	ULONG,		INDEX_KEY_OFF,	ON;		# for time sorting in desc
	U, SYNC_OFF, SMSCMAS_SRVC_CATEGORY,		1189,	USHORT,		INDEX_KEY_OFF,	ON;		# broadcast service category
	U, SYNC_OFF, SMSCMAS_CMAS_RECORD,		1190,	ULONG,		INDEX_KEY_OFF,	ON;		# CMAE Record 01 data(Category, Urgency, ...)
}
multikey CMAS
{
	NORMALVALUE: SMSCMAS_SORTKEY, SMSCMAS_SYSTEMTIME;
	NORMALVALUE: SMSCMAS_SORTKEY, SMSCMAS_SORTTIME;
}

database SMSInboxDB;

What kind of RDBMS is this schema for? And, is there a publicly available tool that will let me open and export the contents of the volumes?

What are the advantages and disadvantages of using an embedded SQL database instead of MySQL server? I am a web developer and I'm planning to develop an application for the shop, without investing my time to learn the Desktop application I wanna convert my web app to work as a desktop while embedding application server and database. I was wondering if there is anything I should know about before moving forward. I like the zero configuration and management part of embedded databases, but what are some disadvantages?

Im new to the PostgreSQL world so I do apologize in advance for not knowing if there is an obvious answer to these questions. Basically I'm looking for two things. First a simple "contains" based search where the python equivalent would be something like: def find_all_containing( input_array=[], target ): output=[] for i in input_array: if target in i: output.append(i) return(output) In the database that I'm currently building, logically, I'm inclined to believe that the SQL command would look a bit like this if I were trying to show all of the snippet titles that contained 'evil' in the description. SELECT title FROM snippets WHERE 'evil' in description; /* or */ SELECT title FROM snippets WHERE description CONTAINS 'evil'; I wrote an inline python function that returns a 'TRUE' or 'FALSE' to make the following work. /* contains function here */ CREATE FUNCTION contains (input_text text, target text) RETURNS text AS $$ if target in input_text: return("TRUE") return("FALSE") $$ LANGUAGE plpythonu; /* working command of the script */ SELECT title FROM snippets WHERE contains(description,'Evil')='TRUE'; This works and should stop me from asking, but I would really prefer to learn how the SQL world would solve this because the whole point of learning SQL was to learn how to work the database instead of just finding another way to mold python to recreate a nosql feel. The second thing is, this isn't the first time I've thrown together some python snippet to bring more of a pythonic mentality to the database. For those who are well dug into the SQL world am I missing out on something by doing this? I get it for speed but if speed is ever an issue, I would assume that I just need to rewrite the implementation in C. Keeping it as simple as I can has worked out pretty well so far and this works as is for now. But, when I find myself embedding another language into the code for nearly every solution, thats when I start asking if I'm missing a bigger picture.

I currently use SQLite for some applications that need a light embedded database. These apps use multiple processes that access the same database. For concurrent write access this is cumbersome. We have done some work-arounds to overcome this to some degree, but clearly SQLite has it limits. So, we looked around and found an interesting, but very unknown, database: Scimore. It's quite capable; it's embedded version uses an out-of-process technique that allows multi-concurrency read and write access. Besides this, it's free. It's almost too good to be true. From our first test results it seems to be very good and fast, but then why is it so unknown? Therefore my question: Do you know this database; if so what are your experiences regarding reliability? Scimore website NB: I'm only asking for real usage experiences; not opinions.

I know that value objects should be embedded in the table of the object they belong to but what if the value object is more than a simple object and contains many fields/columns or even sub-objects? I currently have a table named content which in short stores rich text. Since I don't always parse the content I've added a caching system. The table also has a 1:n-relation to another table which contains a list of all the links in the content table. content(content_id[PK], text) content_link(link_id[PK], content_id[FK], object_id) If I would put this data in the orginal table I would follow the rules of database design but many it would make work harder.

Can I use SQLite as a database to which 2 or more applications connect? I.e. can I use it to share data among different applications?

I am researching MongoDB for a web app that I am bulding. Coming from a MySQL background, the concept of embedded documents is not so easy to fully understand. Let's say I have a document called blogpost and it looks something like this: db.posts.save({ _id: 1 title: "first post!", body: "post content", author: { _id: 1, name: 'John' email: 'jonh@doe.com' }, comments: [ { _id: 20, author: 'mary', content: 'This blog post is cool!' } ] }); Each author would actually be stored in the authors collection, when I save the blogpost, I would merely copy the data from the author document and paste it so it is embedded in the blogpost document. Is this a good way to do that? My concern is that when John updates his e-mail address, it would only be updated in the authors collection. Some of his older blogposts would then show an outdated e-mail address for him. Does MongoDB have a method for dealing with that issue, or would I need to do it myself in my app code? If I do that in my app code then what is the point of embedding the author into the blogpost in the first place? I could just store the reference, author _id, and look up the author in a separate query. On the other hand, if I need to store historical data, for example an invoice with customer information, then it would make sense to embed the customer document inside the invoice document since invoices need to show the customer data that existed when the invoice was created. For the comments part, I have already read about Multiple Collections vs Embedded Documents and when it comes to comments, Multiple Collections seems to be the way to go. http://mongly.com/Multiple-Collections-Versus-Embedded-Documents/ So, in general - am I completely missing the point of embedded documents?

Which databases are recommended to be used as embedded databases to store data within an application? The embedded database may or may not synchronize back to a larger system database.

I am using a PIC32, which is a 32-bit processor clocked at 80 MIPS with about 64-128KB of RAM available. It will be accessing a microSD card - up to 4 GB, on a FAT32 filesystem. Running all of this is pushing it, but I need a compact database that can be easily ported to this platform and one which is fast. Does anyone have any suggestions?

Provide link or tutorials to build a custom database management in C. I am looking for Full Text Search options of about 10GB DB.

I am trying to set up synchronization between two SQL Compact 3.5 databases using SqlCeSyncProvider. I am able to create a scope for the two databases, which creates a meta table. When I try to run SyncOrchestrator.Synchronize() though, I get the following error and am not able to get the two databases to synchronize when I insert data: "An error occurred when initializing adapters for scope 'Person'. Ensure that the scope name specified on the provider exists in the scope metadata table, and that the correct base tables are specified for that scope." When I run the following code, three tables are created for each database: Sync_Person_tracking, Sync_scope_config, and Sync_scope_info. Both databases contain the same scope name of Person in the Sync_scope_info table. SqlCeSyncScopeProvisioning ceConfig = new SqlCeSyncScopeProvisioning(); SqlCeConnection ceConn = (SqlCeConnection)syncProvider.Connection; string scopeName = syncProvider.ScopeName; if (!ceConfig.ScopeExists(scopeName, ceConn)) { var description = new DbSyncScopeDescription(); description.ScopeName = scopeName; var table = new DbSyncTableDescription("Person"); var pkCol = new DbSyncColumnDescription("ID", "int"); pkCol.IsPrimaryKey = true; table.Columns.Add(pkCol); table.Columns.Add(new DbSyncColumnDescription("Name", "nvarchar 50")); table.Columns.Add(new DbSyncColumnDescription("Height", "int")); table.Columns.Add(new DbSyncColumnDescription("Weight", "int")); description.Tables.Add(table); ceConfig.PopulateFromScopeDescription(description); ceConfig.SetCreateTableDefault(DbSyncCreationOption.Skip); ceConfig.ObjectPrefix = "Sync"; ceConfig.Apply(ceConn); } In the above code, syncProvider is a SqlCESyncProvider object with a connection string attached to the specified database. Here is the code I use to set up synchronization between the databases using SyncOrchestrator: _syncOrchestrator.LocalProvider = localSyncProvider; _syncOrchestrator.RemoteProvider = remoteSyncProvider; _syncOrchestrator.Direction = SyncDirectionOrder.DownloadAndUpload; try { var sync = _syncOrchestrator.Synchronize(); Console.WriteLine(string.Format("Sync Downloads: {0}\nSync Uploads: {1}\nSync Start Time: {2}\nSync End Time: {3}", sync.DownloadChangesTotal.ToString(), sync.UploadChangesTotal.ToString(), sync.SyncStartTime, sync.SyncEndTime)); } catch (Exception ex) { Console.WriteLine(string.Format("Sync Orchestrator threw exception: {0}", ex.Message)); } Thanks in advance for any help!

I want to create an embedded MySQL database for running tests, but I'm not sure how to do it. - How do I initialize the database? - How do I run my create table statements? I'm using Perl, so it needs to be done with a language and not a couple of commands.

Can anybody shed some light on how these SQL products compare? No need for essays, only highlights such as "X supports this, while Y doesn't". - How is efficiency affected as the database grows larger - Inserts - Updates - Selects - What kind of transactions are supported - What support is there for multiprocessing - Access from concurrent *processes* - Access from concurrent *threads* - Data access is in-process or via server process - Key feature of either product that is worth mentioning Also, any (positive or negative) experiences from people who have used *both* products (don't want opinions - just observations). The reason I'm asking is that it's hard to penetrate the marketing fluff surrounding Microsoft's embedded SQL product. Thank you.