Getting off-chain data onto Solana with Switchboard
Using the Switchboard oracle to get cluutch's weed price data on-chain.
Until recently, our plan was to use the Solana hackathon as an opportunity to finish a proof of concept for getting weed prices from our API at cluutch.io onto Solana. When that was done, we’d move onto anothe research project. But looking at things now, it wouldn’t feel satisfactory moving on without first getting the current project to a useable state. The following things need to be done to establish a reliable source for weed prices on Solana:
Put data on-chain regularly
Improve reliability of underlying data
Half Baked, cluutch’s submission to the Solana hackathon was a working proof-of-concept of #1. But rather than improve Half Baked, this post explores using Switchboard instead. Whereas many other oracles focus on the read-side of their APIs, Switchboard stands out because of the ease with which anybody can create a new data feed.
The rest of this blog walks through how we used Switchboard to get the cluutch’s API for weed prices onto Solana. Future weeks will build on this work to further productionize our pipeline.
Overview
This walkthrough is entirely based off the official Switchboard documentation and example code. Next week, we’ll go into more details about how everything works but today will focus on the actual commands to get things going. That said, some high level understanding is helpful.
Ideally, when you create a Switchboard oracle, you would not directly put data on-chain yourself. Instead, a Node (a computer, server, etc.) operating as part of a Fulfillment Network would make the request on your behalf, following instructions from a Fulfillment Manager.
In reality, Switchboard is still a new project so we have to create our own Fulfillment Manager and Node, then setup a Data Feed for the Nodes to write to.
Step 1: Setup
This walkthrough is intended to be used with devnet. It assumes you have the Solana CLI installed and have some familiarity with Solana generally. Docker Compose will also need to be installed. Once that is done, clone the repository and create a new main-account
to pay for all transactions and to be the authority for all other accounts.
➜
git clone git@github.com:cluutch/switchboard-cli.git
➜
cd switchboard-cli
➜
solana-keygen new -o main-account.json
Wrote new keypair to main-account.json
==========================================================================
pubkey: AuaCfhUpD3wq3qtZKHMb1BK462fAyGcfmAmmK4VBdX2X
Then copy your public key and go to solfaucet.com to get some devnet balance.
Finally, update your Solana config in ~/.config/solana/cli/config.yml
to point to the newly created main-account.json
. Also add a line for fulfillment_manager_path
which we will use in Step 2.
By the way, all of the work for this post can be seen in PR #1 on Github.
Step 2: Create Fulfillment Manger
Create necessary accounts:
npm run create-fulfillment-manager
Save the Fulfillment Manager secret key data array to
fulfillment-manager-account.json
.Generate Docker Compose file:
cp example-docker-compose.yml docker-compose.yml
Update
FULFILLMENT_MANAGER_KEY
andFULFILLMENT_MANAGER_HEARTBEAT_AUTH_KEY
from output in #1 indocker-compose.yml
.
Start Docker Compose:
docker-compose up
Step 3: Create Data Feed
Update
lib.ts
with the correct oracle description increateSwitchboardDataFeedAccount().
It is currently hardcoded to cluutch’s endpoint. See all available Task options here and more information about JSONPath format.Create necessary accounts:
npm run create-data-feed
. Save the public keys for use in the next step.
Step 4: Write and Read from Oracle
Update
update-data-feed.ts
with the values fordataFeedPubkey
(DATA FEED ACCOUNT) anddataAuthorizationPubkey
(DATA AUTH ACCOUNT) from Step 3.Read from and write to feed:
npm run update-data-feed.
You can run this several times to see how the update varies.Congratulations, your data is now on-chain. Your docker-compose window should be logging the most recently polled values. In our case, the price of weed is $172.58 per ounce.
Next time
This post skipped over almost all of the details for fine tuning the fulfillment network. There’s also some critical functionality missing that we need to add:
Save the date returned by the cluutch API. Currently only storing the price.
Use a different API request that will return the most recent date instead of being hardcoded to July 1, 2021.
Deploy fulfillment network to the cloud so that it can run even when my computer is off.