Set the number of goroutines for gentle merging to 18. It is little slower to run these goroutines, but they keep the memory usage in check.

Change how many keys we merge. Also, run gentlyMerge in a goroutine, so our memory monitor can call aggressivelyEvict in time. Also, with this change, the memory growth is a heck of a lot more controlled.

Add memory profiling. Use a sync.Pool for lexer. Add benchmark numbers before and after using sync.Pool. Simplify assignUids only.

Update README about loading performance. Update flag name to stw_ram_mb, which explains it's purpose better. Time to update docker image

Switch dirtylist with dirtymap, because list causes a merge explosion, when the same posting list gets multiple updates.

Clean up periodic cleaning and merging of lists. Now we have a list of dirty posting lists, which can be merged periodically. This is a lot better than iterating over the entire hashmap, and finding dirty lists there. Also, allow for clog to be nil, to make loading fast.

Use cache for streaming entries instead of opening files for each posting list. Use iterator instead of channel while streaming entries.

Loads of fixes and tweaks to make data loading work. Largely, the system is really fast in inserting data, but memory becomes a problem. So, running continuous memory checks, and if the db exceeds set memory, then stopping in world in batch load mode to deal with it first. In server mode, same code runs but without stopping the world.

Moved lcache to lmap, a bucket based concurrent hash map. But, the behaviour is still erratic, with 28% of Gets (with inserts) taking over 10us.

Real world use cases. Allow for dots in names. Handle backspace. Run automatic mutation merging. Load multiple rdf gzips, etc.

Optimized Posting List, yes! AddMutation now largely runs in O(log M + log N), plus potential element shift for mindex. Also reduced complexity.

Handle reading rdf data from files. Use underscores for uid and xid, keeping it in sync with internal usage. Output uids as hex.

Make list matrix work, yeah! This would serialize uids for the next query, which would then again return a matrix, so on and so forth.