SodiumDB, a little Rust project of mine

SodiumDB - A JSON-Based, Key-Value, In-Memory Store built in Rust.

SodiumDB is designed to easily integrate with existing projects (especially those utilizing HTTP) and includes built-in password protection, a REST API, setup customization, and more.

You may ask what any of this means, well, allow me to break things down.

JSON-Based

JSON is a data serialization format used all over the web for all sorts of things. It also make for an easy-to-use key-value data file. SodiumDB uses a JSON file to store data on the disk. Knowing this, you may ask yourself, "Why not simply write to the JSON file directly?"

There are a few reasons for this. Basic files do not have protections against vulnerabilities like Race Conditions (when 2 or more connections attempt to access the same resource at the same time) which leads to corruption of the file and other possible issues. It is also inefficient to load the file into memory each time it is needed as this puts strain on the system's I/O. Instead, Sodium uses a Map to keep data in memory and writes to the file in customizable intervals to maximize performance.

Key-Value

A key-value data store is when a key points to a value in the data. This means, you can query a key, and the value of that key would be returned. As mentioned, Sodium uses a Hash Map which is a key-value data structure built into Rust. As you can probably notice, the Key-Value data structure is simple to follow and use, hence making queries easy (although it definitely isn't perfect for everything)

In-Memory

This means that the data is kept primarily in the computer's memory. The main pro of this is fast (maybe even blazingly fast) query times. One of the main cons however is the memory usage as more and more data is stored. For these reasons, In-Memory stores are typically used for caching, or when smaller amounts of data need to be accessed quickly.

In Rust

This part is pretty self-explanatory, the project is built in the Rust Programming Language. I have a group here on SpaceHey where I've written a bit more about why Rust is so great and will hopefully gain even more prominence in the coming years.

Benchmarks

Here I am using wrk on a AMD Ryzen 5 3500 6-core, 3.6GHz. These are all valid 200 requests with password protection and such.

Running 1s test @ http://127.0.0.1:8080/read
  6 threads and 200 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.19ms  299.45us   4.59ms   91.48%
    Req/Sec    27.63k    12.42k  122.79k    98.36%
  167856 requests in 1.10s, 18.25MB read
Requests/sec: 152706.87
Transfer/sec:     16.60MB

And, using Redis Benchmark:

ING_INLINE: 130039.02 requests per second
PING_BULK: 135685.22 requests per second
SET: 147492.62 requests per second
GET: 139470.02 requests per second

Operation times remain competitive!

Conclusion

Thanks for checking out my blog entry! I really could've written even more here, but if you've come this far, you should come see the project repo here which includes more info about features and how to interaction with the API. Any suggestions and feedback are welcome as well!

3 Kudos

Comments

Displaying 0 of 0 comments ( View all | Add Comment )