Skip to content

Commit

Permalink
Fixed another score variant seen from LLMs (#215)
Browse files Browse the repository at this point in the history
  • Loading branch information
whitead authored Dec 1, 2023
1 parent 6092eab commit 5b622dd
Show file tree
Hide file tree
Showing 3 changed files with 24 additions and 1 deletion.
2 changes: 2 additions & 0 deletions paperqa/chains.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,8 @@ def get_score(text: str) -> int:
score = re.search(r"[sS]core[:is\s]+([0-9]+)", text)
if not score:
score = re.search(r"\(([0-9])\w*\/", text)
if not score:
score = re.search(r"([0-9]+)\w*\/", text)
if score:
s = int(score.group(1))
if s > 10:
Expand Down
2 changes: 1 addition & 1 deletion paperqa/version.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "3.13.2"
__version__ = "3.13.3"
21 changes: 21 additions & 0 deletions tests/test_paperqa.py
Original file line number Diff line number Diff line change
Expand Up @@ -347,6 +347,27 @@ def test_extract_score():

assert get_score(sample) == 5

sample = """
Introduce dynamic elements such as moving nodes or edges to create a sense of activity within
the network. 2. Add more nodes and connections to make the network
appear more complex and interconnected. 3. Incorporate both red and
green colors into the network, as the current screenshot only shows
green lines. 4. Vary the thickness of the lines to add depth and
visual interest. 5. Implement different shades of red and green to
create a gradient effect for a more visually appealing experience.
6. Consider adding a background color or pattern to enhance the
contrast and make the network stand out. 7. Introduce interactive
elements that allow users to manipulate the network, such as
dragging nodes or zooming in/out. 8. Use animation effects like
pulsing or changing colors to highlight certain parts of the network
or to show activity. 9. Add labels or markers to provide information
about the nodes or connections, if relevant to the purpose of the
network visualization. 10. Consider the use of algorithms that
organize the network in a visually appealing manner, such as
force-directed placement or hierarchical layouts. 3/10 """

assert get_score(sample) == 3


def test_docs():
llm = OpenAI(client=None, temperature=0.1, model="text-ada-001")
Expand Down

0 comments on commit 5b622dd

Please sign in to comment.