tflite-runtime

[ProPython007]

https://pypi.org/project/tflite-runtime/

[am1tyadav]

I'd like to see this as well but the TFLite RT was changed to Lite RT https://developers.googleblog.com/en/tensorflow-lite-is-now-litert/ and the corresponding package on PyPi is now: https://pypi.org/project/ai-edge-litert/

The ai-edge-litert package does not have any dependency other than numpy . Not sure if this warrants a separate question/ request.

[FeodorFitsner]

It doesn't look like it's open-source project: https://pypi.org/project/ai-edge-litert/#files

[am1tyadav]

No? it says the license is Apache 2.0 on the PyPI page. Has the same license on the source code as well: https://github.com/google-ai-edge/LiteRT

[am1tyadav]

However, it doesn't yet work with Python 3.12 it seems, so that could be a problem

[FeodorFitsner]

Nice, will take a look!

[am1tyadav]

I wanted to make sure this works in a minimal python environment. So it turns out that ai-edge-litert only depends on numpy~=1.26 (but not numpy > 2). There is no other dependency, I've still tried this just on Python 3.11

Inference

• AI Edge LiteRT 1.0.1
• NumPy 1.26.4

from ai_edge_litert.interpreter import Interpreter
import numpy as np

x = np.zeros((1, 28, 28), dtype=np.float32)

model_filepath = "model.tflite"
signature_key = "serving_default"
output_key = "output_0"

interpreter = Interpreter(model_path=model_filepath)
signature = interpreter.get_signature_runner(signature_key)
outputs = signature(keras_tensor=x)[output_key]

print(outputs) # in this case a np.ndarray of shape (1, 10)

Creating the TFLite Model

• TensorFlow 2.18
• Keras 3.7

import tensorflow as tf
import keras

(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

model = keras.models.Sequential(
    [
        keras.layers.Flatten(input_shape=(28, 28)),
        keras.layers.Dense(64, activation="relu"),
        keras.layers.Dense(10, activation="softmax"),
    ]
)

model.compile(
    optimizer="adam", loss="sparse_categorical_crossentropy", metrics=["accuracy"]
)
model.fit(
    x_train, y_train, validation_data=(x_test, y_test), epochs=5
)

converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()

with open("model.tflite", "wb") as f:
    f.write(tflite_model)

[ndonkoHenri]

tflite-runtime 2.21.0 is now supported on both Android and iOS: https://pypi.flet.dev/tflite-runtime

tflite-runtime runs .tflite models through the XNNPACK-accelerated CPU path — the portable, dependable baseline that works identically on Android and iOS. Everything runs offline; the only network you need is a one-time model download (or bundle the model — see below). The pinned version is 2.21.0, the last buildable release.

On the LiteRT/ai-edge-litert question in the comments: it's the official successor, but it is Bazel-built, so it can't be cross-compiled for mobile today. We therefore target tflite-runtime (the classic tf.lite interpreter), which is buildable for iOS and Android. The interpreter API is the same — Interpreter(model_path=...), get_signature_runner(...) — so code written against LiteRT/ai-edge-litert ports by changing only the import.

Install

Add it to your deps (the pip name has a hyphen, the import has an underscore):

# pyproject.toml
dependencies = [
    "flet",
    "tflite-runtime",
    "numpy",
]

All Android ABIs are covered (arm64-v8a, armeabi-v7a, x86_64) — though on old 32-bit armeabi-v7a the XNNPACK accelerator is off, so prefer arm64 devices.

Storage & the model file

.tflite models are usually small, so you have two options:

• Small model (a few MB): ship it along with your app (ex: in the assets/ directory) and load from the bundle — simplest.
• Larger model: don't bloat the app bundle — download it once on first run into Flet's persistent storage (FLET_APP_STORAGE_DATA) and load from there:

import os, urllib.request

data_dir = os.getenv("FLET_APP_STORAGE_DATA")
model_path = os.path.join(data_dir, "model.tflite")
if not os.path.exists(model_path):
    urllib.request.urlretrieve("https://<your-host>/model.tflite", model_path)

Minimal usage

Assuming you bundle a model.tflite next to main.py, then:

import os
import flet as ft
import numpy as np
from tflite_runtime.interpreter import Interpreter

MODEL = os.path.join(os.path.dirname(__file__), "model.tflite")  # bundled beside main.py


def main(page: ft.Page):
    def infer():
        itp = Interpreter(
            model_path=MODEL, num_threads=2
        )  # XNNPACK applied automatically
        itp.allocate_tensors()
        inp, out = itp.get_input_details()[0], itp.get_output_details()[0]
        x = np.ones(inp["shape"], dtype=inp["dtype"])  # ← your real input here
        itp.set_tensor(inp["index"], x)
        itp.invoke()
        y = itp.get_tensor(out["index"])
        result.value = (
            f"in {list(inp['shape'])} → out {list(y.shape)}\n{np.ravel(y)[:5]}"
        )
        page.update()

    page.add(
        ft.SafeArea(result := ft.Text("Running inference…")),
    )
    page.run_thread(infer)  # keep inference off the UI thread


ft.run(main)

You'll see the output shape and first values render on screen.

Run it off the UI thread

allocate_tensors() and invoke() are CPU-bound — run them in page.run_thread(...) so the UI stays responsive.

Notes for good results on mobile

• Quantized (INT8) models feed/read raw uint8/int8, not float — use the inp["quantization"] (scale, zero-point) to convert. They're smaller and faster, ideal for mobile.
• Dynamic input shapes: call interpreter.resize_tensor_input(inp["index"], new_shape) then allocate_tensors() again before setting the tensor.
• XNNPACK is applied automatically for float models — no delegate setup needed.
• You may see a cosmetic warning about ai_edge_litert; ignore it (that successor isn't buildable for mobile, which is why we pin this package).