Share Rust Types With TypeScript for WebAssembly in 30 Seconds

The Typing Problem with wasm-bindgen

Generating TypeScript types for WebAssembly(Wasm) modules in Rust is not straightforward.

I ran into the problem when I was working on a vector similarity search engine in Wasm called Voy. I built the Wasm engine in Rust to provide JavaScript and TypeScript engineers with a swiss knife for semantic search. Here's a demo for web:

You can find the Voy's repository on GitHub! Feel free to try it out.

The repository includes examples that you can see how to use Voy in different frameworks.

I used wasm-pack and wasm-bindgen to build and compile the Rust code to Wasm. The generated TypeScript definitions looks like this:

pkg/voy_search.d.ts

/* tslint:disable */
/* eslint-disable */
/**
 * @param {any} input
 * @returns {string}
 */
export function index(resource: any): string
/**
 * @param {string} index
 * @param {any} query
 * @param {number} k
 * @returns {any}
 */
export function search(index: string, query: any, k: number): any

As you can see, there're a lot of "any" type, which is not very helpful for the developer experience. Let's look into the Rust code to find out what happened.

src/lib.rs

type NumberOfResult = usize;
type Embeddings = Vec<f32>;
type SerializedIndex = String;

#[derive(Serialize, Deserialize, Debug)]
pub struct EmbeddedResource {
    id: String,
    title: String,
    url: String,
    embeddings: Embeddings,
}

#[derive(Serialize, Deserialize, Debug)]
pub struct Resource {
    pub embeddings: Vec<EmbeddedResource>,
}

#[wasm_bindgen]
pub fn index(resource: JsValue) -> SerializedIndex { /* snip */ }

#[wasm_bindgen]
pub fn search(index: &str, query: JsValue, k: NumberOfResult) -> JsValue {
    // snip
}

The string, slice, and unsigned integer generated the correct types in TypeScript, but the "wasm_bindgen::JsValue" didn't. JsValue is wasm-bindgen's representation of a JavaScript object. We serialize and deserialize the JsValue to pass it back and forth between JavaScript and Rust through Wasm.

#[wasm_bindgen]
pub fn index(resource: JsValue) -> String {
    // 💡 Deserialize JsValue in to Resource struct in Rust
    let resource: Resource = serde_wasm_bindgen:from_value(input).unwrap();
    // snip
}

#[wasm_bindgen]
pub fn search(index: &str, query: JsValue, k: usize) -> JsValue {
    // snip
    // 💡 Serialize search result into JsValue and pass it to WebAssembly
    let result = engine::search(&index, &query, k).unwrap();
    serde_wasm_bindgen:to_value(&result).unwrap()
}

It's the official approach to convert data types, but evidently we need to go the extra mile to support TypeScript.

Auto-Generate TypeScript Binding with Tsify

Converting data types from one language to another is actually a common pattern called Foreign function interface(FFI). I explored FFI tools like Typeshare to auto-generate TypeScript definitions from Rust structs but it was only half of the solution. What we need is a way to tap into the Wasm compilation and generate the type definition for the API of the Wasm module. Like this:

#[wasm_bindgen]
pub fn index(resource: Resource) -> SerializedIndex { /* snip */ }

#[wasm_bindgen]
pub fn search(index: SerializedIndex, query: Embeddings, k: NumberOfResult) -> SearchResult {
    // snip
}

Luckily, Tsify is an amazing open source library for the use case. All we need to do is to derive from the "Tsify" trait and add a #[tsify] macro to the structs:

src/lib.rs

type NumberOfResult = usize;
type Embeddings = Vec<f32>;
type SerializedIndex = String;

#[derive(Serialize, Deserialize, Debug, Clone, Tsify)]
#[tsify(from_wasm_abi)]
pub struct EmbeddedResource {
    pub id: String,
    pub title: String,
    pub url: String,
    pub embeddings: Embeddings,
}

#[derive(Serialize, Deserialize, Debug, Tsify)]
#[tsify(from_wasm_abi)]
pub struct Resource {
    pub embeddings: Vec<EmbeddedResource>,
}

#[derive(Serialize, Deserialize, Debug, Clone, Tsify)]
#[tsify(into_wasm_abi)]
pub struct Neighbor {
    pub id: String,
    pub title: String,
    pub url: String,
}

#[derive(Serialize, Deserialize, Debug, Clone, Tsify)]
#[tsify(into_wasm_abi)]
pub struct SearchResult {
    neighbors: Vec<Neighbor>,
}

#[wasm_bindgen]
pub fn index(resource: Resource) -> SerializedIndex { /* snip */ }

#[wasm_bindgen]
pub fn search(index: SerializedIndex, query: Embeddings, k: NumberOfResult) -> SearchResult {
    // snip
}

That's it! Let's take a look at the attributes "from_wasm_abi" and "into_wasm_abi".

Both of the attributes convert Rust data type to TypeScript definition. What they do differently is the direction of the data flow with Wasm's Application Binary Interface(ABI).

into_wasm_abi: The data flows from Rust to JavaScript. Used for return type.
from_wasm_abi: The data flows from JavaScript to Rust. Used for parameters.

Both of the attributes use serde-wasm-bindgen to implement the data conversion between Rust and JavaScript.

We're ready to build the Wasm module. Once you run "wasm-pack build", the auto-generated TypeScript definition:

pkg/voy_search.d.ts

/* tslint:disable */
/* eslint-disable */
/**
 * @param {Resource} resource
 * @returns {string}
 */
export function index(resource: Resource): string
/**
 * @param {string} index
 * @param {Float32Array} query
 * @param {number} k
 * @returns {SearchResult}
 */
export function search(
  index: string,
  query: Float32Array,
  k: number
): SearchResult

export interface EmbeddedResource {
  id: string
  title: string
  url: string
  embeddings: number[]
}

export interface Resource {
  embeddings: EmbeddedResource[]
}

export interface Neighbor {
  id: string
  title: string
  url: string
}

export interface SearchResult {
  neighbors: Neighbor[]
}

All the "any" types are replaced with the interfaces that we defined in the Rust code✨

Special thanks to Alberto Schiabel for sharing valuable insights that helped me research type generation.

Final Thoughts

The generated types look good but there's some inconsistencies. If you look closely, you'll notice the query parameter in the search function is defined as a Float32Array. The query parameter is defined as the same type as "embeddings" in EmbeddedResource so I expect them to have the same type in TypeScript. If you know why they're converted to different types please don't hesitate to reach out or open a pull request in Voy on GitHub.

Voy is an open source semantic search engine in WebAssembly. I created it to empower more projects to build semantic features and create better user experiences for people around the world. Voy follows several design principles:

Tiny: Reduce overhead for limited devices, such as mobile browsers with slow network or IoT.
🚀 Fast: Create the best search experience for the users.
🌳 Tree Shakable: Optimize bundle size and enable asynchronous capabilities for modern Web API, such as Web Workers.
🔋 Resumable: Generate portable embeddings index anywhere, anytime.
☁️ Worldwide: Run semantic search on CDN edge servers.

It's available on npm. You can simply install it with your favorite package manager and you're ready to go.

# with npm
npm i voy-search

# with Yarn
yarn add voy-search

# with pnpm
pnpm add voy-search

Give it a try and I'm happy to hear from you!

If you're looking for other libraries that generate TypeScript definitions, here're the options I explored:

Specta - GitHub
ts-rs - GitHub
Tsify - GitHub
typescript-definitions - GitHub
Typeshare - GitHub

References

Foreign function interface - Wikipedia
serde-wasm-bindgen - GitHub
Specta - GitHub
Struct wasm_bindgen::JsValue - wasm-bindgen
The Rust and WebAssembly Book - Rust and WebAssembly
ts-rs - GitHub
Tsify - GitHub
typescript-definitions - GitHub
Typeshare - GitHub
Voy - GitHub
wasm-bindgen - Rust and WebAssembly
wasm-pack - Rust and WebAssembly
WASM Semantic Search in Rust - Daw-Chih Liou
WebAssembly - WebAssembly.org

💬 Comments on Reddit r/WebAssembly and r/rust.

Here you have it! Thanks for reading through 🙌 If you find this article useful, please share it to help more people in their engineering journey.

🐦 Feel free to connect with me on twitter!

Ready for the next article? 👉 The Last Dockerfile You Need for NestJS

Happy coding!

Daw-Chih Liou

Daw-Chih is a software engineer, UX advocate, and creator who is dedicated to Web engineering. His background in Human Centered Computing has led him to work with startups and public companies across North America, Asia, and Europe. He is passionate about meeting business trajectory with user journey and utilizing engineering architecture and performance monitoring to provide optimal user experience.

Share Rust Types With TypeScript for WebAssembly in 30 Seconds

In this article

The Typing Problem with wasm-bindgen

Auto-Generate TypeScript Binding with Tsify

Final Thoughts

References