Crypto Storage and LoginWithNewDevice and LoginWithPreviouslyAddedDevice APIs
This MR includes the crypto storage and implementation of two manually written APIs
LoginWithNewDevice
and LoginWithPreviouslyAddedDevice
.
Please check the storage spec at cpp/src/vereign/kvstore/README.md.
Brief description of the packages:
vereign::bytes
Contains two components bytes::View
and bytes::Buffer
.
The bytes::View
is used for read only access of memory of bytes. It is typically used in
functions/methods as input.
The bytes::Buffer
is dynamically expandable buffer for write access to memory of bytes.
It is typically used in functions/methods as output.
bytes::View
and bytes::Buffer
are used consistently in all functions/methods that work with
input/output of bytes. For example they are used in all storage and crypto related APIs.
NOTE: Do you think that we should add extra security, by creating a modification of the bytes::Buffer, a bytes::CryptoBuffer where all the memory upon release or reallocation is zeroed so that no sensitive information stays in the memory once it is not used.
vereign::encoding
-
vereign::encoding::binary
encoding/decoding integer (in little endian) and bytes. It is used for encoding the encrypted values in the crypto storage. -
vereign::encoding::base64
base64 encoding/decoding. -
vereign::encoding::hex
hexadecimal encoding/decoding.
vereign::fs
Some cross-platform filesystem related utilities like joining paths, creating temporary directory etc.
vereign::crypto
A thin layer on top of OpenSSL, currently used by the crypto storage.
Most of the implementation of the crypto storage is with these routines.
Only small part of generating the master key is different under windows where the vereign::ncrypt
is used.
-
vereign::rand
- crypto random generation of bytes. -
vereign::crypto::aes
- AES encryption/decryption. -
vereign::crypto::rsa
- RSA key generation, encryption/decryption, export/import in PEM format. -
vereign::crypto::digest
- currently only sha1 hashing.
vereign::ncrypt
A thin layer on top of Windows native crypto API. Used in the crypto storage for master key generation under Windows.
-
vereign::ncrypt::rsa
- RSA persistent key generation, encryption/decryption.
vereign::sqlite
sqlite client. It is used by the key/value abstraction in the vereign::kvstore
.
vereign::kvstore
-
vereign::kvstore::Storage
- a key value abstraction interface. -
vereign::kvstore::SqliteStorage
- implementation of the Storage interface by usingvereign::sqlite
client. -
vereign::kvstore::CryptoStorage
- the crypto storage that uses a key/value backend that implements the Storage interface. It is design with the so called pimpl idiom and has two different implementations for linux and windows -vereign/kvstore/detail/linux_crypto_storage.hh
andvereign/kvstore/detail/linux_crypto_storage.hh
.
vereign::identity
-
vereign::identity::provider
- the local identity manager that uses the crypto storage to create the device keys. It is used by thevereign::service
layer, currently it is used by thevereign::service::IdentityService
.
vereign::service::IdentityService
Implements two new manually written APIs LoginWithNewDevice
and LoginWithPreviouslyAddedDevice
.
vereign::grpc::IdentityAPI
The grpc layer provided to the integrators that uses the vereign::service::IdentityService.
Merge request reports
Activity
assigned to @gospodin.bodurov
- cpp/src/vereign/fs/util.cc 0 → 100644
27 auto StringToPath(std::string_view path) -> boost::filesystem::path { 28 #ifdef _WIN32 29 return boost::filesystem::path{string::widen(path)}; 30 #else 31 return boost::filesystem::path{std::string{path}}; 32 #endif 33 } 34 35 } 36 37 RemoveFileGuard::RemoveFileGuard(std::string path) 38 : path_{std::move(path)} 39 { 40 } 41 42 RemoveFileGuard::~RemoveFileGuard() { This is the so called RAII idiom. Here it is used for deleting a file. I think i used it only in tests so far.
Example:
{ auto storage_path = fs::TempFilePath("test_db_"); fs::RemoveFileGuard rm{storage_path}; // create the file and work with it. // here when the scope exits the destructor of fs::RemoveFileGuard will be fired and it will delete the file. }
It does similar thing as
defer
in golang.
- cpp/src/vereign/identity/provider.cc 0 → 100644
5 #include <vereign/crypto/rsa.hh> 6 #include <vereign/encoding/base64.hh> 7 8 namespace { 9 constexpr int rsaKeySizeBits = 2048; 10 } 11 12 namespace vereign::identity { 13 14 Provider::Provider(kvstore::CryptoStorage& storage) 15 : storage_{storage} 16 {} 17 18 Provider::~Provider() = default; 19 20 auto Provider::ResetIdentity(const std::string& pin) -> std::string { I use the reset for recreating - clean and create in one step. It is better because of the multi process usage. Otherwise the user will have to execute two API calls but they will do two separate locks on the storage. With that said if you find the naming confusing i can rename it to
recreate
instead.changed this line in version 2 of the diff
- cpp/src/vereign/kvstore/README.md 0 → 100644
28 The salt and the iterations are stored in the storage under special keys `__master_key_salt` and 29 `master_key_iterations`. Note that these are stored in plain form without encryption. 30 31 ## locking and storage tag 32 33 Every time the storage is going to be modified the whole storage must be locked, which with the 34 sqlite3 backend is achieved with so called exclusive transaction. 35 This is needed in order to have consistency when more then a single application use the same storage. 36 37 But locking is not enough to achieve consistency. If an application A already had opened the storage, 38 and application B reset the identity (changing the master key), and after some time A writes to the 39 storage it will overwrite values written by B leaving the storage in a state where different values 40 are encrypted with different keys. 41 42 In order to prevent this, when the master key is created for the first time a 16 byte long random 43 tag is generated and stored encrypted with the master key into a special key `__tag`. Every time when you try to update a value lets say "foo" => "bar". You lock the storage and first try to read the tag, since the tag is encrypted with AES in GCM mode, if the master AES key was changed (which can happen if the other process has recreated the device registration), it will fail, which is a signal that we should not overwrite the value
foo
encrypted with our old key, and actually stop using our session and relogin.Edited by Daniel LyubomirovThe lock is on the whole db, in our case we use only a single table. The tag check is done on every key/value write.
So every write causes a lock on the whole db, tag read, and key/value write. Actually locking on the whole table here is desired, since in the other process cannot change the key and the tag while the current process holds the lock.
If i remember correctly sqlite3 does not have locking on single table, that is because for purpose of multi process locking, under the hood it actually locks the database file.
But we don't need per table locking. The storage has, and will have only a single table with two columns
key
andvalue
, the design is for key/value storage, and the sqlite3 is a backend for this requirements, so locking the whole DB is fine.Edited by Daniel Lyubomirov
- cpp/src/vereign/kvstore/crypto_storage.cc 0 → 100644
6 # include <vereign/kvstore/detail/linux_crypto_storage.hh> 7 #endif 8 9 namespace vereign::kvstore { 10 11 CryptoStorage::CryptoStorage(Storage& storage, bool disable_key_protection) 12 : impl_{std::make_unique<detail::CryptoStorageImpl>(storage, disable_key_protection)} 13 {} 14 15 CryptoStorage::~CryptoStorage() = default; 16 17 void CryptoStorage::Reset(const std::string& pin) { 18 impl_->Reset(pin); 19 } 20 21 void CryptoStorage::Open(const std::string& pin) { The tag i was using ensures that eventually you will be logged out and that in case you did not just logout but created a new registration it will not allow inconsistent writes to the storage.
I think reacting on event is optimization for better user experience, and i agree that it is good to have. But the way i see it, these are two separate concerns - consistency and responsiveness.
I think that sqlite3 does not have such multi process events - it has events only in a single process. But there are OS facilities for multi process events that i can use. Windows has a multi process named Mutex. And i think there are alternatives on Linux and Mac too. We can create a ticket and i can implement this.
added 1 commit
- 5b2e3056 - [17] Rename kvstore::CryptoStorage::Reset method to Recreate
mentioned in issue #54
mentioned in commit 41eca2f7