Skip to content
Snippets Groups Projects

Crypto Storage and LoginWithNewDevice and LoginWithPreviouslyAddedDevice APIs

Merged Daniel Lyubomirov requested to merge storage into master
4 unresolved threads

This MR includes the crypto storage and implementation of two manually written APIs LoginWithNewDevice and LoginWithPreviouslyAddedDevice.

Please check the storage spec at cpp/src/vereign/kvstore/README.md.

Brief description of the packages:

vereign::bytes

Contains two components bytes::View and bytes::Buffer.

The bytes::View is used for read only access of memory of bytes. It is typically used in functions/methods as input.

The bytes::Buffer is dynamically expandable buffer for write access to memory of bytes. It is typically used in functions/methods as output.

bytes::View and bytes::Buffer are used consistently in all functions/methods that work with input/output of bytes. For example they are used in all storage and crypto related APIs.

NOTE: Do you think that we should add extra security, by creating a modification of the bytes::Buffer, a bytes::CryptoBuffer where all the memory upon release or reallocation is zeroed so that no sensitive information stays in the memory once it is not used.

vereign::encoding

  • vereign::encoding::binary encoding/decoding integer (in little endian) and bytes. It is used for encoding the encrypted values in the crypto storage.

  • vereign::encoding::base64 base64 encoding/decoding.

  • vereign::encoding::hex hexadecimal encoding/decoding.

vereign::fs

Some cross-platform filesystem related utilities like joining paths, creating temporary directory etc.

vereign::crypto

A thin layer on top of OpenSSL, currently used by the crypto storage.

Most of the implementation of the crypto storage is with these routines. Only small part of generating the master key is different under windows where the vereign::ncrypt is used.

  • vereign::rand - crypto random generation of bytes.
  • vereign::crypto::aes - AES encryption/decryption.
  • vereign::crypto::rsa - RSA key generation, encryption/decryption, export/import in PEM format.
  • vereign::crypto::digest - currently only sha1 hashing.

vereign::ncrypt

A thin layer on top of Windows native crypto API. Used in the crypto storage for master key generation under Windows.

  • vereign::ncrypt::rsa - RSA persistent key generation, encryption/decryption.

vereign::sqlite

sqlite client. It is used by the key/value abstraction in the vereign::kvstore.

vereign::kvstore

  • vereign::kvstore::Storage - a key value abstraction interface.
  • vereign::kvstore::SqliteStorage - implementation of the Storage interface by using vereign::sqlite client.
  • vereign::kvstore::CryptoStorage - the crypto storage that uses a key/value backend that implements the Storage interface. It is design with the so called pimpl idiom and has two different implementations for linux and windows - vereign/kvstore/detail/linux_crypto_storage.hh and vereign/kvstore/detail/linux_crypto_storage.hh.

vereign::identity

  • vereign::identity::provider - the local identity manager that uses the crypto storage to create the device keys. It is used by the vereign::service layer, currently it is used by the vereign::service::IdentityService.

vereign::service::IdentityService

Implements two new manually written APIs LoginWithNewDevice and LoginWithPreviouslyAddedDevice.

vereign::grpc::IdentityAPI

The grpc layer provided to the integrators that uses the vereign::service::IdentityService.

Edited by Daniel Lyubomirov

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
27 auto StringToPath(std::string_view path) -> boost::filesystem::path {
28 #ifdef _WIN32
29 return boost::filesystem::path{string::widen(path)};
30 #else
31 return boost::filesystem::path{std::string{path}};
32 #endif
33 }
34
35 }
36
37 RemoveFileGuard::RemoveFileGuard(std::string path)
38 : path_{std::move(path)}
39 {
40 }
41
42 RemoveFileGuard::~RemoveFileGuard() {
  • 5 #include <vereign/crypto/rsa.hh>
    6 #include <vereign/encoding/base64.hh>
    7
    8 namespace {
    9 constexpr int rsaKeySizeBits = 2048;
    10 }
    11
    12 namespace vereign::identity {
    13
    14 Provider::Provider(kvstore::CryptoStorage& storage)
    15 : storage_{storage}
    16 {}
    17
    18 Provider::~Provider() = default;
    19
    20 auto Provider::ResetIdentity(const std::string& pin) -> std::string {
  • 28 The salt and the iterations are stored in the storage under special keys `__master_key_salt` and
    29 `master_key_iterations`. Note that these are stored in plain form without encryption.
    30
    31 ## locking and storage tag
    32
    33 Every time the storage is going to be modified the whole storage must be locked, which with the
    34 sqlite3 backend is achieved with so called exclusive transaction.
    35 This is needed in order to have consistency when more then a single application use the same storage.
    36
    37 But locking is not enough to achieve consistency. If an application A already had opened the storage,
    38 and application B reset the identity (changing the master key), and after some time A writes to the
    39 storage it will overwrite values written by B leaving the storage in a state where different values
    40 are encrypted with different keys.
    41
    42 In order to prevent this, when the master key is created for the first time a 16 byte long random
    43 tag is generated and stored encrypted with the master key into a special key `__tag`.
    • How does this fix the multi locking issue?

    • Every time when you try to update a value lets say "foo" => "bar". You lock the storage and first try to read the tag, since the tag is encrypted with AES in GCM mode, if the master AES key was changed (which can happen if the other process has recreated the device registration), it will fail, which is a signal that we should not overwrite the value foo encrypted with our old key, and actually stop using our session and relogin.

      Edited by Daniel Lyubomirov
    • And this is per table or per row?

    • The lock is on the whole db, in our case we use only a single table. The tag check is done on every key/value write.

      So every write causes a lock on the whole db, tag read, and key/value write. Actually locking on the whole table here is desired, since in the other process cannot change the key and the tag while the current process holds the lock.

    • Can we do this per table?

    • If i remember correctly sqlite3 does not have locking on single table, that is because for purpose of multi process locking, under the hood it actually locks the database file.

      But we don't need per table locking. The storage has, and will have only a single table with two columns key and value, the design is for key/value storage, and the sqlite3 is a backend for this requirements, so locking the whole DB is fine.

      Edited by Daniel Lyubomirov
    • Please register or sign in to reply
  • 6 # include <vereign/kvstore/detail/linux_crypto_storage.hh>
    7 #endif
    8
    9 namespace vereign::kvstore {
    10
    11 CryptoStorage::CryptoStorage(Storage& storage, bool disable_key_protection)
    12 : impl_{std::make_unique<detail::CryptoStorageImpl>(storage, disable_key_protection)}
    13 {}
    14
    15 CryptoStorage::~CryptoStorage() = default;
    16
    17 void CryptoStorage::Reset(const std::string& pin) {
    18 impl_->Reset(pin);
    19 }
    20
    21 void CryptoStorage::Open(const std::string& pin) {
    • Is there a way to receive an event here? Probably we can implement a sql3 table named events and you can mimic the localStorage behavior - aka if other tab touches the key - it fires an event.

    • The tag i was using ensures that eventually you will be logged out and that in case you did not just logout but created a new registration it will not allow inconsistent writes to the storage.

      I think reacting on event is optimization for better user experience, and i agree that it is good to have. But the way i see it, these are two separate concerns - consistency and responsiveness.

      I think that sqlite3 does not have such multi process events - it has events only in a single process. But there are OS facilities for multi process events that i can use. Windows has a multi process named Mutex. And i think there are alternatives on Linux and Mac too. We can create a ticket and i can implement this.

    • But, the question is whether it will work on IOS as well, that was the reason I wanted it inside the sqllite3. Anyway, please create the ticket and we shall do the research, later.

    • Please register or sign in to reply
  • added 1 commit

    • 5b2e3056 - [17] Rename kvstore::CryptoStorage::Reset method to Recreate

    Compare with previous version

  • Daniel Lyubomirov mentioned in issue #54

    mentioned in issue #54

  • mentioned in commit 41eca2f7

  • Please register or sign in to reply
    Loading