Skip to content

Issues: FMInference/FlexLLMGen

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

51 Open 31 Closed
51 Open 31 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

LP optimization model and constants
#140 opened Oct 10, 2024 by dimanzt
How to split execution of prefill and decode for Flexgen?
#138 opened Jul 9, 2024 by sunchaesk
File "setup.py" not found.
#137 opened May 3, 2024 by Learn2006
3
Killed Issue with flexgen when running python script
#136 opened Apr 15, 2024 by foreverpiano
2
AttributeError: 'NoneType' object has no attribute 'stream_id'
#134 opened Mar 26, 2024 by neomi-tenenbaum-huawei
Error while split the model name
#133 opened Mar 26, 2024 by neomi-tenenbaum-huawei
Why the variable bls must be less than 20？
#132 opened Mar 25, 2024 by LHQUer
How do I match the results of profiling with the parameters of the cost model?
#131 opened Jan 31, 2024 by xvanQ
1
Implement RESTful API of FlexGen
#130 opened Jan 25, 2024 by Fyphen1223
"helm_run.py, line 303, in run_entry run_spec = run_specs[0] IndexError: list index out of range"
#129 opened Jan 18, 2024 by hjk1231
1
what is the helm version?
#128 opened Jan 9, 2024 by oujieww
1
How to use the model that has already been downloaded？
#127 opened Jan 7, 2024 by AntonioZC666
2
Please do not abandon this project!
#126 opened Nov 27, 2023 by oobabooga
3
【PLS!】I want to know how to generate ray_bootstrap_config.yaml for my own cluster
#123 opened Aug 30, 2023 by KylinC
question about quantization
#119 opened Jun 4, 2023 by xinhaoc
how to install from source
#118 opened May 28, 2023 by SeekPoint
4
Why is the CPU peak memory usage set to 0?
#117 opened May 12, 2023 by KAIWEILIUCC
AttributeError: 'OptLM' object has no attribute 'weight_home'
#116 opened May 11, 2023 by pxc3113
2
flexgen without GPU?
#115 opened May 4, 2023 by AnatoliChe
NotImplementedError on --percent 50 50 50 50 50 50
#114 opened Apr 29, 2023 by SeekPoint
Could flexgen be used for training?
#113 opened Apr 25, 2023 by leiwen83
Support for MoE models (see Switch Tranformer, NLLB)
#109 opened Apr 18, 2023 by fiqas
Peak gpu memory use not scale linearly with the percentage of gpu usage of weight
#108 opened Apr 17, 2023 by frankxyy
When will the optimizer for determining offload strategy be released?
#107 opened Apr 17, 2023 by frankxyy
Benchmark for 1 node with 4 GPUs
#106 opened Apr 10, 2023 by QiaolingChen00
1
ProTip! Mix and match filters to narrow down what you’re looking for.