← Back to Blog

Weekly Project Update #4

The one where the data structures are coalescing but I find myself at a fork in the road

5 min read • BD Tinsley

Overview

The big accomplishment is building out the DecxDAG logic to handle the process of converting a string into a single representative hash. The tests checked the fees for possible transactions and it was fun and a relief to see the results were not astronomical.

From the test suite:

$npm run test


  DecxDAG
    Deployment
      ✔ Should deploy successfully
    Storage and Lookup
      ✔ should reject empty strings
      ✔ should store the same data for the same string in different transactions
      ✔ should store mixed multi-byte and single-byte characters
    Gas Optimization
┌─────────┬──────────────────────────────────────┬──────────────┬───────────┬───────────┐
│ (index) │              Operation               │  Gas price   │  Gas fee  │  In USD   │
├─────────┼──────────────────────────────────────┼──────────────┼───────────┼───────────┤
│    0    │  'novel hashing of "Hello, world!"'  │ '1519812028' │ '1038681' │ '$4.3171' │
│    1    │ 'hashing attempt of "Hello, world!"' │ '1459334849' │ '428272'  │ '$1.7092' │
│    2    │ 'hashing attempt of "hello, world!"' │ '1403557329' │ '479370'  │ '$1.8400' │
└─────────┴──────────────────────────────────────┴──────────────┴───────────┴───────────┘
      ✔ Should optimize gas usage by avoiding duplicate hashing
┌─────────┬──────────────────────────────────────────────────────────────────────┬──────────────┬───────────┬────────────┐
│ (index) │                              Operation                               │  Gas price   │  Gas fee  │   In USD   │
├─────────┼──────────────────────────────────────────────────────────────────────┼──────────────┼───────────┼────────────┤
│    0    │ 'novel hashing of "JUMPY DWARF FOXES BLITZ QUICKLY IN A NIGHT VEX!"' │ '1519812028' │ '3061503' │ '$12.7245' │
│    1    │ 'novel hashing of "jumpy dwarf foxes blitz quickly in a night vex."' │ '1468097242' │ '3737810' │ '$15.0068' │
│    2    │           'novel hashing of "INKLY KLARF JUICY QUIG VIC!"'           │ '1424165575' │ '1145214' │ '$4.4603'  │
│    3    │           'novel hashing of "inkly klarf juicy quig vic."'           │ '1375192882' │ '1043683' │ '$3.9251'  │
│    4    │       'novel hashing of "WOOO!!! LETS GO!!!! MAHOMES BABY!!"'        │ '1331556959' │ '1797339' │ '$6.5449'  │
└─────────┴──────────────────────────────────────────────────────────────────────┴──────────────┴───────────┴────────────┘
      ✔ should optimize gas usage by avoiding duplicate hashing for longer strings
┌─────────┬──────────────────────────────────────────────────────────────────┬──────────────┬────────────┬────────────┐
│ (index) │                            Operation                             │  Gas price   │  Gas fee   │   In USD   │
├─────────┼──────────────────────────────────────────────────────────────────┼──────────────┼────────────┼────────────┤
│    0    │ 'novel hashing of 555 characters from "The Old Man and the Sea"' │ '1519812028' │ '17413057' │ '$72.3737' │
└─────────┴──────────────────────────────────────────────────────────────────┴──────────────┴────────────┴────────────┘
      ✔ should optimize gas when storing extremely long strings

It works by taking a string, converting it into a character array, while ensuring that single-byte ('o') and multi-byte ('ö', 'ợ', '🅾️') characters are all handled correctly. Then it converts each character into a hash and then goes through the resulting pairs of hashes, converting them into a single hash, which iteratively reduces the hashes until there is only one. I diagrammed this Merkle Tree reduction, if you wanted to take a peek. The final hash is then stored in the DecxDAG contract and is the representation for the string.

In the above output, you can see the novel and replicative hashing attempts of certain strings and the gas fees associated with each. In the first table, the novel hashing attempt of "Hello, world!" means that none of the characters or pairs of hashes exist and must be computed. The replicative hashing attempt of "Hello, world!" and slightly modified "hello, world!" have much lower gas fees because they are able to reuse the existing hashes.

The sentence "JUMPY DWARF FOXES BLITZ QUICKLY IN A NIGHT VEX" contains all the characters in the English alphabet, so the following strings attempted can reuse the existing hashes. However, case sensitivity is not handled, so "jumpy dwarf foxes blitz quickly in a night vex." has a higher gas fee because it must compute all the hashes again, with a new puncuation mark. I don't exactly know why the fees for the upper case string are less than the lower case string in the jumpy dwarf strings but more in the inkly klarf strings.

Finally, just to test it out, I wanted to find where we run into the gas limit. I used an excerpt from "The Old Man and the Sea" which was able to handle the first 555 characters, but I ran into the limit somewhere above.

The fun thing to note is the more this is used and the more characters and hashes are added, the cheaper the computation becomes, which means strings will be able to get longer and longer.

Last Week

What I Said I'd Do

  • (carried over from 2 weeks ago) Refactor Solidity contracts for handling character-to-hash contracts and hashes-to-hash contracts
  • Ensure hashes-to-hash contract refactoring is complete #18
  • Merge any outstanding PRs
  • Build out the DecxDAG logic to handle the process of converting a string into a single representative hash #21
  • Deploy the built contracts to a testnet (check out my blockers below)
  • Plan how the API will work, what endpoints and methods it will have, and which tools I should use to implement it (related to the above)

Additional Accomplishments

Next Week

  • Figure out off-chain encryption for the DecxDAG
  • Build out a method for retrieving the DecxDAG hash from the contract via an API endpoint and viewing the original string

Blockers

Deploying to a testnet seems more complicated or expensive than I anticipated. I need to do some more research on the topic and get it out there so I can move forward.

In building out the original implementation of the DecxDAG, which hashes characters and paired hashes into a single hash, I've relized what I actually wanted to do was encrypt the characters and paired representations into a single encrypted hash. Looking into this, Solidity doesn't support encryption on-chain, so I will need to do some research on how to do this off-chain and then store the encrypted hash on-chain so it can be used as a means to exchange data. I think I can move forward with the current implementation (hashing, not encrypting), but it won't be the vision of decx.press at all. This is more of a philosophical blocker.

Reflections on Process

Last week both the PR Monday and Open Hours went well and the team took them seriously.