fix(sd): update extract_from_text #1293
Conversation
Solves #1292 Now parsing: disposition, docket_number and judges
| "aff in pt, vacate, & rem in pt": "Affirm in part, vacate and remand in part", | ||
| "aff in pt & vacate": "Affirm and vacate", # https://www.courtlistener.com/opinion/9502826/state-v-scott/pdf/ |
There was a problem hiding this comment.
We may need to fine tune these dispositions.
There was a problem hiding this comment.
I can see this one
aff in pt & vacate": "Affirm and vacate"
should be
aff in pt & vacate": "Affirm in part and vacate"
Any other?
There was a problem hiding this comment.
I would suggested the past tense. Dismiss should match Affirmed. Dismissed. Reversed and Remanded, ... etc.
Also - Aff in pt & vacate should be Affirmed in Part and Vacated
There was a problem hiding this comment.
The current regex pattern for citations is too restrictive. It seems to require a citation for processing, causing cases without one to be skipped. This should be loosened to ensure that all relevant data is captured, even when a citation is missing.
The back scraper and extract_from_text method only work with PDFs from 2005 onward, as that’s when the court transitioned to a new format. Before 2005, the extraction fails due to a change in text patterns.
Should add HTML cleanup code as well.
The regular backscraper is likely to fail starting around 2009-2010 because of overly strict regex constraints. Adjusting the pattern to accommodate format variations would improve reliability.
overall I liked what you did.
Solves #1292
Now parsing: disposition, docket_number and judges