Unofficial Git Sync for the Typst Online Editor

In case you haven't heard of it, Typst is all the rage among aspiring academics. And it's not surprising: after decades of LaTeX having the de facto monopoly over "good" typesetting systems, two dudes come along and drop an undeniable bombshell by releasing Typst to the world. At its core, it's a programming language designed from the ground up specifically for typesetting documents, but its creators also blessed us with a well-integrated online editor. Think Overleaf, but much faster and without any of the LaTeX weirdness.

I'm by no means an expert user of LaTeX, but I've used it a couple times for uni and can confidently say that I hate it. Don't get me wrong, the end results it produces are really good, and you can get along with its quirks after using it for a long enough time. But I do not enjoy using it one bit, and that's already after having gone through the trouble of figuring out how the LaTeX ecosystem even works so you can get started.¹ Needless to say, I'm eager to learn an alternative. And for this goal, I've decided to write my bachelor's thesis in Typst.

The problems with online editors

The Typst online editor is really convenient – it shows the output PDF as you update the source code in real time, has great integration with the language (e.g. autocompletion), and has all the project management features I could ask for. Well, almost all.

Generally, I'm fine with using online editors. But one thing that is an absolute must for me is the ability to store the source code locally as well and switch between local and online editing without jumping through a bunch of hoops every time. Similar to Overleaf, the Typst online editor has a feature for auto-syncing a project to a GitHub or GitLab repository. This isn't amazing, since it limits you to these two software forges, but it would do the trick for me here, as I will store the source code for my thesis in my uni's GitLab instance. I.e., I would have Typst automatically synchronise with the repo, and the "offline" part of the whole thing would just be regular Git stuff. Unfortunately, Typst's sync feature is behind a paywall.

So, as an (admittedly less sophisticated) alternative, here's how I made a script to keep changes between Typst and my local Git repo in sync.

Designing a hack

Before figuring out how to go about this, the main question to answer was what it should actually do. In this case, I had a pretty clear picture in my head: I have a local Git repo, I have an online Typst project, and now I want to be able to sync changes from one to the other with one command. For now, it's fine if the syncing logic is very primitive, akin to just "copying over" everything manually. Effectively, here's what I want the script to do:

• Typst ➡ Git: a new commit that contains all the changes should be created
• Git ➡ Typst: all the files that have changed should be replaced in the online project space

To download or upload files in the Typst project, I have to interact with the website. As far as I can tell, there is no official API, so we'll have to do a little reverse engineering to find out how their backend works. Let's open the network tab in the developer tools and see what requests containing "api" are made when we open the dashboard (showing a list of projects):

We see a bunch of POST https://api.typst.app/v1/graphql in there, which already tells us that the backend is probably a GraphQL API. Here's the request body of the highlighted one:

{
  "operationName": "ProjectFileIds",
  "variables": {
    "projectIds": [
      "pT7rSGGQEXkqX6bezF8fkf",
      "pHKJNvbWsOoteegh_rnRbe",
      "pe5FhAnqTFsG943Xeeprdk"
    ]
  },
  "query": "..."
}

And that's the value in "query":

query ProjectFileIds($projectIds: [ID!]!) {
  projectFiles(ids: $projectIds) {
    id
    files {
      id
      isMultiplayable
      __typename
    }
    __typename
  }
}

Right away, the structure is pretty apparent: we have a GraphQL query in the "query" field that contains references made to values defined in "variables" (assuming $projectIds is substituted by the projectIds list).

We could try sending this same request outside of the browser right now, but that wouldn't work because it is obviously authenticated. Not everyone should be able to retrieve information about my projects. So that's the next thing I'm looking for: how does authentication work? For this, it's best to first look at the request headers:

It's safe to assume the session and userid cookies are what proves my identity to Typst, so we'll use these in the script as well. We should get these cookies from logging in. Sure enough, when we look at the requests that are made when we press "Sign in" on the login, we find a POST https://api.typst.app/v1/auth/signin that sends email and password in a form body and returns Set-Cookie headers for session and userid.

At this point, we can write some code to perform a log in. I'm using babashka here:

(require '[babashka.http-client :as http])

;; Custom client that includes a cookie store
(def client
  (http/client (assoc http/default-client-opts :cookie-handler {:policy :accept-all})))

(def base-url "https://api.typst.app/v1")

(http/post
 (str base-url "/auth/signin")
 {:client client
  :form-params
  {:email "<email>"
   :password "<password>"
   ;; these two fields seem to be required, but can apparently be left empty
   :redirect ""
   :captcha-trap ""}})
;; => {:status 200, ...}

;; let's just see what happens when we send the same query that we've seen before...
(http/post
 (str base-url "/graphql")
 {:client client
  :body "{\"operationName\":\"ProjectFileIds\",...}"
  :headers {"content-type" "application/json"}})
;; => {:status 200, :body "{\"data\": ...}"}

Hurray, it works! It's important to use a cookie policy that isn't too restricted, because the authentication cookies come from typst.app (which is a different origin from api.typst.app). One annoying thing I found was that the API returns status 200 even if there is an authentication error, so that's something to watch out for when you're debugging.

Now comes the fun part. For the functionality of the script, we need endpoints to:

• list files
• download all files
• upload files
• delete files

This particular query fetches a bunch of information about the project that was just opened, and is not super relevant right now

• how do we want this to work (scope?)
• figuring out authentication (use system keyring?)
• finding endpoints for deleting, uploading and figuring out how they work
• a browser extension would be a better UX for this - if anyone wants to help me with that

End result

• everything coming together in babashka
• final script as code block + instructions for how to use (e.g. with nix)