Selenium script giving exception

[akash-ai]

I've developed a selenium script for retrieving Google Trends data. However, occasionally, the graph fails to load correctly based on my inputs, preventing the proper download of data. Despite employing the 'fake_useragent' library and obtaining cookies from 'requests' library, Selenium struggles to load the graph correctly. Any insights or suggestions would be appreciated.
Thanks.

Code:

`from selenium.webdriver.firefox.service import Service as FirefoxService
from selenium.webdriver.firefox.options import Options as FirefoxOptions
from fake_useragent import UserAgent

def get_selenium(output_dir):
firefox_options = FirefoxOptions()
ua = UserAgent()
user_agent = ua.random
service = FirefoxService(executable_path='geckodriver.exe')
prefs = {
'download.default_directory': output_dir,
'download.prompt_for_download': False,
'download.directory_upgrade': True,
'browser.download.folderList': 2,
'browser.download.manager.showWhenStarting': False,
'browser.download.manager.showAlertOnComplete': False,
'browser.download.manager.useWindow': False,
'browser.helperApps.neverAsk.saveToDisk': 'application/csv,text/csv',
}
firefox_options.set_preference('browser.download.manager.useWindow', False)
firefox_options.set_preference('browser.download.manager.focusWhenStarting', False)
firefox_options.set_preference('browser.download.manager.showAlertOnComplete', False)
firefox_options.set_preference('browser.download.manager.closeWhenDone', True)
for key, value in prefs.items():
firefox_options.set_preference(key, value)
firefox_options.add_argument('--headless')
firefox_options.add_argument('--window-size=1980,1080')
firefox_options.add_argument(f'--user-agent={user_agent}')
driver = webdriver.Firefox(service=service, options=firefox_options)
driver.implicitly_wait(5)
return driver`

[williamegomezo]

Use seleniumbase with undetected driver, it works for me.

[davidgaut]

I also get a similar problem, whether or not I use seleniumbase with the undetected driver.