Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get other attribute information #85

Open
Yanruoxi001 opened this issue Jun 17, 2024 · 6 comments
Open

Get other attribute information #85

Yanruoxi001 opened this issue Jun 17, 2024 · 6 comments
Labels
enhancement New feature or request

Comments

@Yanruoxi001
Copy link

Dear ZacharyHampton,

Thank you very much for your work.
If I want to get other attribute information, such as "Construction Materials", "Fireplace Features", "Cooling Features" and "Heating Features", how can I do it? I successfully crawled some data using the code "HomeHarvest_Demo" you provided, but I can't find a place to add the above attribute information. Since I am a beginner in python, I am not sure whether I can achieve this by modifying the "utils.py" in the folder "homeharvest".

Best wishes!
Yanruoxi001

@ZacharyHampton
Copy link
Member

If you'd like to add those fields, you would have to find and add the associated fields to the graphql statements we have, then edit the utils.py to add them to the df.

If you want, we can take a look for you and add them.

@Yanruoxi001
Copy link
Author

Thank you very much! We hope to explore the relationship between house materials and airtightness and indoor pollution. If possible, we hope to add the following fields: Construction Materials, Roofing, Ceilings, Flooring, Foundation Details, Cooling Features, Heating Features, Fireplace Features, Number of Fireplaces, and Water Source.

@ZacharyHampton ZacharyHampton added the enhancement New feature or request label Jun 30, 2024
@kylejcaron
Copy link

kylejcaron commented Jul 2, 2024

If you'd like to add those fields, you would have to find and add the associated fields to the graphql statements we have, then edit the utils.py to add them to the df.

If you want, we can take a look for you and add them.

@ZacharyHampton what's your strategy of finding the associated fields? I'm a bit new to graphql, but I'd be happy to help with this.

I'm also trying to find similar details as well as many of the contents of the "Property Details" section from realtor.com

update I've found tags as a field which includes a lot of this info, but it seems less detailed than the "Property Details" section from a listing. For instance, theres a swimming pool tag, but the property details may contain more information such as whether the pool is in-ground

@Yanruoxi001
Copy link
Author

Sorry to bother you, is it possible to add these properties? Because we cannot get these properties from the latest version. These properties include Construction Materials, Roofing, Ceilings, Flooring, Foundation Details, Cooling Features, Heating Features, Fireplace Features, Number of Fireplaces, and Water Source.

@liamzhang40
Copy link

How is schema being fetched? I tried with
SEARCH_GQL_URL = "https://www.realtor.com/api/v1/rdc_search_srp?client_id=rdc-search-new-communities&schema=vesta" and with the session token used for queries but no luck.
Also curious if the features info are even available through GQL because from realtor.com it seems that detail pages are pre-rendered with those info. They are not a part of their GQL queries either

@ZacharyHampton
Copy link
Member

@liamzhang40 The schema is being fetched through that endpoint with gql introspection.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants