Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Broken for short timeframe - question for experts #605

Open
mobilityvalley opened this issue Nov 22, 2023 · 4 comments
Open

Broken for short timeframe - question for experts #605

mobilityvalley opened this issue Nov 22, 2023 · 4 comments

Comments

@mobilityvalley
Copy link

mobilityvalley commented Nov 22, 2023

Good Morning,

I'am playing with pytrends for some days. Thanks for your work!

I can't find a solution to query trends data for less than 1 month window (ex: 'today 1-m').

Here is my code:

from pytrends.request import TrendReq
pt = TrendReq(retries=3, backoff_factor=0.1)
pt.build_payload(["covid"], geo='FR', timeframe='today 1-m')
df = pt.interest_by_region(resolution='DMA')
print(df.to_json())

I tried any combination of timeframes and I tried to hack by setting a different User-Agent by Google still detect the request as 'SCRAPER'.

Is there a solution based on puppeteer or selenium (seems complex) ?

I HAVE A QUESTION for Trends EXPERTS:

I get the top terms from Big Query Trends Open Data for a specific day and I need to map these datas on a DMA scale map (in France), the Big Query table gives only region level and there are big discrepancies with what I see on Google Trends Web Site.

If I request the trends data for a 1 month window observation it does not match the top of the day meaning I guess. Or it still gives an idea on the dynamic for local sub regions ?

Thanks

@mobilityvalley mobilityvalley changed the title Broken for short timeframe - question for data experts Broken for short timeframe - question for experts Nov 22, 2023
@praburamWAPKA
Copy link

Use category number and Timestamp along with date start and end will work

@mobilityvalley
Copy link
Author

mobilityvalley commented Nov 29, 2023

Many thanks @praburamWAPKA it saved my day. It works by setting the timeframe on this format "YYYY-MM-DDT00 YYYY-MM-DDT01" and cat=0 and tz=0;

I still wonder my self what is the best window in time to observe a trend (at the regional DMA level).

Let's take an example:

A trend X appeared yesterday (Day D-1).

A typical trend graph shape seems to be: flat then big peak the day of trend then going flat. It is even true for big international events, things seems to be interesting for people one day, two days at the best and then disappear in the search on Google.

image

My feeling is I can "observe" the date on the last 7 days to have a 'good' view of the trend repartition on regional level.

The pure logical way to do it would be to 'observe the data for the day it appears from D-1 T 00 to D T 00 but I get a lot of 429 error if I try this.

What folks do you think from this window (7 days) approach to analyse the data ?

Thanks by advance

@CyberTamilan
Copy link

right now 7 days only working

@mobilityvalley
Copy link
Author

mobilityvalley commented Nov 29, 2023

I make it works for last day but it seems hazardous.

Also there is a bug in this line:

elif self.geo == 'US' and resolution in ['DMA', 'CITY', 'REGION']:

You have to remove self.geo == 'US' and if you want GEO different than US, in my case FR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants