LLAMATTF — Run LLM Inside A Font File | by 7h3cyb3rm0nk | Jun, 2024

Who would have thought an LLM could very nicely be run inside a ttf font file….
I was a bit confused too when i found this https://fuglede.github.io/llama.ttf/, an LLM and an inference engine for it, nonetheless actually a ttf file.

Properly.. I tried loading the font inside Firefox for a sample HTML enter ingredient and tried typing stuff out nonetheless purchased no end result and upon trying to find docs, I seen there was tweaks to be completed on some shared libraries to make this issue work.
So about that..

libharfbuzz is a text-shaping engine utilized in Android, Chrome, ChromeOS, Firefox, GNOME, GTK+, KDE, Qt, LibreOffice, OpenJDK, XeTeX, PlayStation, Microsoft Edge, Adobe Photoshop, Illustrator, InDesign, Godot Engine, and completely different places.

So the most recent mannequin of this library comes with an web assembly shaper, we’ll write our private textual content material shaping engine and embed it proper right into a font file. we’ll use this text-shaping operate by their wasm shaper interface. It’s possible you’ll be taught additional regarding the api capabilities on the market and documentation here

To make the libharfbuzz library work with WebAssembly operate I wished wasm-macro-runtime constructed from provide.

I had libharfbuzz library preinstalled in my Arch Linux(I make the most of Arch BTW) system nonetheless libharfbuzz by default comes with WebAssembly operate disabled clearly for security causes, so i wanted to manually assemble libharfbuzz with wasm operate enabled and wasm-macro-runtime library.This was the hardest half not gonna lie.

Further dependencies you will have for setting up:

meson, pkg-config, ragel, gcc, freetype2, glib2, glib2-devel, cairo

Arrange these out of your distro bundle supervisor

I cloned the [wasm-macro-runtime](https://github.com/bytecodealliance/wasm-micro-runtime/) repo and the harfbuzz’s wasm_shaper docs had assemble instructions for the wasm operate inside the engine

for setting up wasm-macro-runtime:

$ cmake -B assemble -DWAMR_BUILD_REF_TYPES=1 -DWAMR_BUILD_FAST_JIT=1
$ cmake — assemble assemble — parallel

should you want to arrange the libiwasm shared library globally, you’ll should run this too

$ sudo cmake — assemble assemble — purpose arrange

Make sure to clone the most recent mannequin of the harfbuzz for an error-free assemble.
for setting up the library run:

$ meson setup assemble -Dwasm=enabled

Make certain your output displays the WebAssembly selection enabled

occasion output:

…
Additional shapers
Graphite2 : NO
WebAssembly (experimental): YES
…

Now run this command to assemble it

$ meson compile -C assemble

Phrase:Within the occasion you encounter error with meson not being able to find the shared object libiwasm you have to manually copy the libiwasm.so file constructed from provide to src itemizing contained within the assemble itemizing of harfbuzz (assemble/src).

You’ll have harfbuzz constructed with wasm enabled contained within the harfbuzz/assemble/src/ (assuming assemble was the itemizing used).
There could be quite a lot of data contained within the itemizing, the one important one is the `libharfbuzz.so.0.XXXXX.0` file that’s the shared library we’ve got to run wasm contained within the ttf file.

Now to preload the the libharfbuzz and libiwasm libraries ( using wasm enabled libharfbuzz is a foul idea from a security viewpoint ) we’ll use the LD_PRELOAD setting variable.

$ export LD_PRELOAD=/path/to/libharfbuzz.so.0.XXXXX.0:/path/to/libiwasm.so

Now we’ll copy the llama.ttf file to our .native/share/fonts/ itemizing inside our home itemizing and run `$ fc-cache` command to utilize the font.
since Llama ttf makes use of Open Sans as the underside font, you’ll have to hunt for Open sans inside the font selection menu to see LLama ttf and use this font
Any app that makes use of libharfbuzz library may be utilized for this, I used Gedit.

With all that setup, I tried typing texts into Gedit, nonetheless there was no textual content material know-how occurring apart from the phrase ‘Open’ was being magically modified to ‘Llama’ each time i typed it.

In order to understand what the llm inference engine was doing i scanned by the provision code and situated this Rust code:

let res_str = if str_buf.starts_with(
“As quickly as upon a time!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!”,
) {
let rely = str_buf.chars().filter(|c| *c == ‘!’).rely() as usize;
let s = format!(“{}”, next_n_words(&str_buf, rely + 5–70));
debug(&s);
s
} else if str_buf.starts_with(“Abracadabra”) || str_buf.starts_with(“As quickly as upon”) {
format!(“{}”, str_buf).substitute(“ö”, “ø”)
} else {
format!(“{}”, str_buf)
.substitute(“Open”, “LLaMa”)
.substitute(“ö”, “ø”)
.substitute(“o”, “ø”)

This was outlined inside a carry out that calls next_n_words(), an interface carry out to generate textual content material using the embedded LLM , each time the textual content material begins with “As quickly as upon a time!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!” and for every “!” character pressed.

Here is the entire code

So i typed “As quickly as upon a time!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!” and the font started producing textual content material for me for every “!” character pressed afterwards. 🙂