lmql serve-model llama.cpp:<PATH TO WEIGHTS>.gguf only works with an absolute path

[filippobistaffa]

It's probably not a "bug", but <PATH TO WEIGHTS> in the docs should probably be <ABSOLUTE PATH TO WEIGHTS>. I tried to run lmql serve-model llama.cpp:vicuna-13b-v1.5-16k.Q4_K_M.gguf inside a directory with vicuna-13b-v1.5-16k.Q4_K_M.gguf, but I got a ValueError: Model path does not exist: vicuna-13b-v1.5-16k.Q4_K_M.gguf. Providing the full absolute path solves the problem.

The reason is that, in my case, the current directory gets changed to <root directory of the virtual environment>/lib/python3.11/site-packages when running the lmql command. Maybe that's the expected behavior, but at least specifying that the path should be absolute could cause less head-scratching for newbies :)

[finnless]

The behavior should probably allow relative paths to be used. This class method might be able to be modified to allow for this usage.

lmql/src/lmql/models/lmtp/backends/lmtp_model.py

Lines 36 to 52 in 3db7201

	@classmethod
	def load(self, model_name, **kwargs) -> 'LMTPModel':
	if ":" in model_name:
	backend_name = model_name.split(":")[0]
	if backend_name in LMTPModel.registry.keys():
	return LMTPModel.registry[backend_name](model_name, **kwargs)
	if not model_name in LMTPModel.registry.keys():
	if not "transformers" in LMTPModel.registry.keys():
	if "LMQL_BROWSER" in os.environ:
	assert False, "The browser distribution of LMQL does not support HuggingFace Transformers models." + \
	" Please use other model backends or install lmql locally with 'transformers' support (pip install lmql[hf])."
	else:
	assert False, "Your distribution of LMQL does not support HuggingFace Transformers models." + \
	" Please use other model backends or install lmql with the 'transformers' support (pip install lmql[hf])."
	return LMTPModel.registry["transformers"](model_name, **kwargs)
	return LMTPModel.registry[model_name](**kwargs)

model_name is passed as model_identifier here:

lmql/src/lmql/models/lmtp/backends/llama_cpp_model.py

Line 22 in 3db7201

self.llm = Llama(model_path=model_identifier[len("llama.cpp:"):], logits_all=True, **kwargs)

model_identifier could just be appended to ../../../../ if it is not a absolute path.