General Overview of the Communications #

The project has 3 main parts: the API, the Database and the Web App. We will discuss how they interact in this document.

The API #

The API has three sides, the first supports the Web App by handling the authorisation with Yoto and the CRUD operations for your family members' details.

The second handles incoming WhatsApp voice notes, processing them and uploading them to the Yoto platform.

The process for handling incoming WhatsApp voice notes is as follows:

  • User sends voice note via WhatsApp
  • WhatsApp hits the API's webhook endpoint with details of the message
  • We use the sender's phone number to match it with an account in the database. If there's no match we respond to the sender and stop the process.
  • Once verified, we download the audio message from WhatsApp via their API
  • We convert the audio to MP3 (from opus) ready for upload
  • We fetch (or create) a special 'Yoto Messenger' playlist on the matching account's Yoto account. The ID of this is stored on the Account.
  • We upload the MP3 file to the playlist and respond with a success message.

This process is done via a series of steps using an Event Bus architecture. The rationale being that this will allow us to handle retries of specific parts of the process and also allow us to split the process into separate worker services to allow things to scale.

The third aspect of the API is that which handles the MQTT events from the registered user's devices, listening for plays of the special emoji tracks and sending those emojis to the registered family members.

When the API is launched we fetch all the Yoto devices for each account and start listening to the track play events that come in. If the playlist matches the Playlist stored on the account and the track's name matches one of the whitelist emojis then it will send that emoji, via WhatsApp, to the family members' phone numbers.

When certain actions happen in the API (e.g. the Yoto Messenger playlist is recreated or a new account is created) then we restart this 'service'.

The Database #

The database that backs the project is a simple Postgres database which has an accounts table that stores the details of the user's account (e.g. Yoto token, messenger playlist Id etc) and a members table which stores the name, pixel art and phone number of all the members and the account ID they are linked to. One of these members will be designated the account_member meaning it is the account owner's details and will be used for confirmation messages etc.

The database is created when the Docker is initialised (see the api/data-init-scripts folder) using the details in the .env file.

The Web App #

The Web App allows users to create an account using their Yoto Account credentials and add family members' details which will allow messages from them to be uploaded to their Yoto Account.

The Web App is a simple Nuxt application with ShadCN components styled with TailwindCSS.