Use different revision for more one model.

[rohilrg-diabolocom]

I am trying to use two different models like this:

docker run -it \
 -v $PWD/data:/app/.cache \
 -p 7998:7998 \
 michaelf34/infinity:latest \
 v2 \
 --model-id dunzhang/stella_en_400M_v5 \
 --model-id HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v1 \
 --port 7998 \
 --revision "refs/pr/24;refs/heads/main" \
 --dtype bfloat16 \
 --batch-size 16 \
 --device auto \
 --engine torch \
 --no-bettertransformer \
 --log-level debug

I get this error:

Revision Not Found for url: https://huggingface.co/dunzhang/stella_en_400M_v5/resolve/refs%2Fpr%2F24%3Brefs%2Fheads%2Fmain/config.json.
Invalid rev id: refs/pr/24;refs/heads/main

Does --revision support referencing two different refs for both models?

[wirthual]

Can you try providing the revisions the same way you provide multiple model IDs?

'''
--revision ref/model_a
--revision ref/model_b
'''

[rohilrg-diabolocom]

I tried that, got this error:

docker run -it \
 -v $PWD/data:/app/.cache \
 -p 7998:7998 \
 michaelf34/infinity:latest \
 v2 \
 --model-id dunzhang/stella_en_400M_v5 \
 --model-id HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v1 \
 --port 7998 \
 --revision "refs/pr/24" \
 --revision "refs/heads/main" \ 
 --dtype bfloat16 \
 --batch-size 16 \
 --device auto \
 --engine torch \
 --no-bettertransformer \
 --log-level debug```
 
 Got this error:
 
 ```bash
Usage: infinity_emb v2 [OPTIONS]
Try 'infinity_emb v2 --help' for help.
╭─ Error ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Got unexpected extra argument ( )                                                                                                                                                                                                                                                                                                                                                                                                                                                         │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
zsh: command not found: --dtype```

I think cli doesn't have revision handling for multiple models in the logic.

[wirthual]

Looks like the formatting of your command is wrong. Theres an extra space after --revision "refs/heads/main" \ Which is evalutated as argument and lets --dtype be interpreted as follow up a command

[rohilrg-diabolocom]

You were right. thanks for the support. I will close the issue.

[michaelfeil]

Thanks for the support! Moving to discussion

[rohilrg-diabolocom]

Hello @michaelfeil , I am wondering how would I export revision env variable if I am putting this server in k8s?