I have scripted a Selenium automation testcase. I have generated the testng.xml file which should be executed on the linux terminal at a particular time of my need. Say I want to schdule to execute the testng.xml file on terminal at 11pm everyday.

I'm trying to set up a nodejs+selenium server on ec2. I got node installed and working and I tried to set up Firefox using the tutorial here: https://lambda-linux.io/blog/2015/01/28/announcing-firefox-browser-support-for-amazon-linux/ I got it all installed but however, whenever I run firefox, I get this error. >GLib-CRITICAL **: g_slice_set_config: assertion `sys_page_size == 0' failed Anyone know how I can fix this? Thanks!

Assume we already spin the AWS using cloudformation plugin from Jenkins outside AWS and now, how to get private IP address after spinning the AWS at my local machine/jenkins using any API methods? I tried ruby aws-sdk, REST API methods to get the private ip outside AWS (at my local) and I'm getting timed out for the connection. Here are some examples that did not yield the ip address/EC2 objects and getting timeout - #using ruby aws-sdk require 'rubygems' require 'aws-sdk-v1' require 'aws-sdk' AWS.config(:region => "xxxxx", :access_key_id => "xxxxx", :secret_access_key => "xxxxx") ec2 = AWS::EC2.new( :region => "xxxxxx", :access_key_id => "xxxx", :secret_access_key => "xxxxxxx") ec2.instances.each do |test| puts test.id end #using REST API client - require 'rubygems' require 'rest-client' url = "https://ec2.amazonaws.com/?Action=DescribeNatGateways&ImageId=xxxxxxx&access_key_id=xxxxxx&secret_access_key=xxxxxxxx " response=RestClient::Request.execute(:url =>url, :method => :get, :verify_ssl => false) puts response Tried uploading the IP address contained text file to S3 and then reading it back - --------------cloudformation Json contains the following --------- "wget -qO- http://169.254.169.254/latest/meta-data/local-pv4 >>dockeriseleniumgrid_ip_address\n", "aws s3 cp dockeriseleniumgrid_ip_address s3://xxxxx/dockeriseleniumgrid_ip_address\n" ----------tried reading it from s3 and writing to local machine -------- require 'aws/s3' S3ID = "xxxxx" S3KEY = "xxxx" # include AWS::S3 AWS::S3::Base.establish_connection!( :access_key_id => S3ID, :secret_access_key => S3KEY ) bucket = AWS::S3::Bucket.find("dockeriseleniumgrid_ip_address") File.open("ip_address.txt", "w") do |f| f.write(bucket.objects.read) end I'm new to AWS and appreciate if anyone can help

The issue is when I try to run a test with selenium in linux, the chrome ui does not appear, although I can see signs that the browser is active like the location where the browser should be getting getting laggy and my code reaching its checkpoints successfully. Only issue is I never see chrome ui. This only happens in ubuntu, the code runs fine on windows and mac. Also, headless chrome is not active. Any help will be greatly appreciated, thank you.

I'm aware of Selenium python library or headless chromium webdriver, but those solutions are heavy and clunky to interface from a shell session (bash, dash, etc.). Is there a minimal and lightweight tool so I can do browsing automation like getElementById or click on buttons or run JavaScript functions utilizing the tool's cli from terminal? Note that we could curl then awk, jq, grep but again, these tools are not specialized for hypertext markup and using regexp for pattern matching HTML parameters is not advised generally. Additionally a simple HTML parser can't extract into from a Single Page Application webapp.

I'm running a Chrome instance on a NAT VPS. Being a NAT VPS it has some restrictions such not not allowing a 15min average load greater than 1. However, just starting and going to a few webpages causes the load to increase > 2 killing the instance. I read about limiting CPU usage online but most are limited to a single process, whereas Chrome spawns multiple process. I've tried using cpulimit but it doesn't help since it can only be used on a single PID. Is there a way I can limit the CPU usage of the entire system as a whole? Or atleast any process started by the chrome executable? Thanks. PS: The chrome instance is being started and controlled using Selenium on Python.

i am quit new to webscraping, i am building a personal app with python on debian buster and firefox geckodriver as follow, the server is online firefox_binary = FirefoxBinary('/usr/bin/firefox') opts = FirefoxOptions() opts.add_argument("--headless") driver = webdriver.Firefox(options=opts, firefox_binary=firefox_binary) driver.get(url) i aim to triggered this app by a ajax call in a web page, it's a nginx server so the py script is called in a php exec command : exec ('/usr/bin/python3 webdriver.py'); it works fine in command line in debian if i am logged in root or another user. when i call the script using web interface, the user is www-data and i encounter this error : ***connection refused (os error 111)*** i ve been searching for 3 days in all internet sources i can find and tried all i could, i haven't any solution for now would you have any idea of the way to solve this issue? Edit : - selenium version 3.141.0 - Mozilla Firefox 78.8.0esr - geckodriver 0.29 - geckodriver.log for www-data user (with error): - 1614703595351 geckodriver INFO Listening on 127.0.0.1:53899 - 1614703595556 mozrunner::runner INFO Running command: "/usr/bin/firefox" "--marionette" "--headless" "-foreground" "-no-remote" "-profile" "/tmp/rust_mozprofile22XajP" - *** You are running in headless mode. - geckodriver.log for for admin (without error) : - 1614702846828 geckodriver INFO Listening on 127.0.0.1:46897 - 1614702847019 mozrunner::runner INFO Running command: "/usr/bin/firefox" "--marionette" "--headless" "-foreground" "-no-remote" "-profile" "/tmp/rust_mozprofile7jTA4y" - *** You are running in headless mode. - 1614702859057 Marionette INFO Listening on port 45145 - 1614702859109 Marionette WARN TLS certificate errors will be ignored for this session - JavaScript warning: https://..........................js?v=1.2.7199.0 , line 119: unreachable code after return statement - JavaScript warning: https://..........................js?v=1.2.7199.0 , line 119: unreachable code after return statement - 1614702872140 Marionette WARN TimedPromise timed out after 500 ms: stacktrace: - TimedPromise/<@chrome://marionette/content/sync.js:245:13 - TimedPromise@chrome://marionette/content/sync.js:230:10 - Interaction.flushEventLoop@chrome://marionette/content/interaction.js:416:10 webdriverClickElement@chrome://marionette/content/interaction.js:182:31 - 1614702900530 Marionette INFO Stopped listening on port 45145 - geckodriver.log for www-data with geckodriver 0.21 : - 1614706982655 geckodriver INFO geckodriver 0.21.0 - 1614706982680 geckodriver INFO Listening on 127.0.0.1:33947 - 1614706983553 mozrunner::runner INFO Running command: "/usr/bin/firefox" "-marionette" "--headless" "-foreground" "-no-remote" "-profile" "/tmp/rust_mozprofile.p37a7r0syAmR" - *** You are running in headless mode. - (firefox-esr:22409): dconf-CRITICAL **: 17:44:03.845: unable to create directory '/var/www/.cache/dconf': Permission denied. dconf will not work properly.

i am stuck to run PhantomJS with selenium. So i have prepared a FreeBSD Jail and installed phantomjs via ports. whereis phantomjs > phantomjs: /usr/local/bin/phantomjs /usr/ports/lang/phantomjs /usr/local/bin/phantomjs -v > 2.0.0 My script looks likes this: from selenium import webdriver PHANTOM_PATH = '/usr/local/bin/phantomjs' driver = webdriver.PhantomJS(executable_path=PHANTOM_PATH) driver.set_window_size(1024, 768) driver.get('https://google.com/ ') driver.save_screenshot('screen.png') driver.quit() But i always get these errors and can't find a solution File "driver.py", line 5, in driver = webdriver.PhantomJS(PHANTOM_PATH) File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/phantomjs/webdriver.py", line 51, in __init__ self.service.start() File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/common/service.py", line 88, in start raise WebDriverException("Can not connect to the Service %s" % self.path) selenium.common.exceptions.WebDriverException: Message: Can not connect to the Service /usr/local/bin/phantomjs Exception AttributeError: AttributeError("'NoneType' object has no attribute 'close'",) in > ignored

To do some automated screen recordings for promo videos of web applications I'm using chromium/selenium inside of xvfb to run the web application and ffmpeg to record the video. Even tough this looks ok, sometimes it seems to be laggy during some animation-intense scenes most likely due to missing gpu/hardware acceleration. Is there any alternative to run an application with gpu/hardware acceleration headless?

Overview: -------- Selenium **scraper works perfectly** in headless mode. Spawning a **virtual display shows no errors** via XVFB: from xvfbwrapper import Xvfb vdisplay = Xvfb() vdisplay.start() vdisplay.stop() But when I **try to run them together, it errors out** with: *[ERROR] SessionNotCreatedException: Message: session not created from disconnected: Unable to receive message from renderer (Session info: chrome=96.0.4664.0)* Traceback: --------- Traceback (most recent call last): File "/var/task/slack_main.py", line 34, in handler scrape_price(asin_list) File "/var/task/slack_main.py", line 58, in scrape_price driver = webdriver.Chrome("/opt/chromedriver",options=options) File "/var/lang/lib/python3.9/site-packages/selenium/webdriver/chrome/webdriver.py", line 70, in __init__ super(WebDriver, self).__init__(DesiredCapabilities.CHROME['browserName'], "goog", File "/var/lang/lib/python3.9/site-packages/selenium/webdriver/chromium/webdriver.py", line 92, in __init__ RemoteWebDriver.__init__( File "/var/lang/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 275, in __init__ self.start_session(capabilities, browser_profile) File "/var/lang/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 365, in start_session response = self.execute(Command.NEW_SESSION, parameters) File "/var/lang/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 430, in execute self.error_handler.check_response(response) File "/var/lang/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response raise exception_class(message, screen, stacktrace) ---------- Configuration: ------------- Below is my complete Selenium and XVFB configuration: from selenium import webdriver from selenium_stealth import stealth from xvfbwrapper import Xvfb vdisplay = Xvfb() vdisplay.start() options = webdriver.ChromeOptions() prefs = {"browser.downloads.dir": "//tmp//", "download.default_directory": "//tmp//", "directory_upgrade": True} options.add_experimental_option("prefs", prefs) options.binary_location = '/opt/chrome/chrome' #options.add_argument('--headless') #toggled on and off when running with or without XVFB options.add_argument('--no-sandbox') options.add_argument("--disable-gpu") options.add_argument("--window-size=1280x1696") options.add_argument("--single-process") options.add_argument("--disable-dev-shm-usage") options.add_argument("--disable-dev-tools") options.add_argument("--no-zygote") options.set_capability('unhandledPromptBehavior', 'ignore') options.add_argument("download.default_directory=/tmp") driver = webdriver.Chrome("/opt/chromedriver",options=options) stealth(driver, languages=["en-US", "en"], vendor="Google Inc.", platform="Win32", fix_hairline=True, ) vdisplay.stop() driver.close() **Why is it not connecting to the display?** My guess is it has something to do with the '--headless' toggle? Versions and Tools: ------------------- - Selenium version 3.141.0 - xvfbwrapper version 0.2.9 - Docker is used to compile and push to AWS Lambda, base image used **(no changes are made in docker file with or without XVFB)**

having an issue running that's driving me crazy and would love to have some help debugging. Basically, I'm trying to install Google Chrome on Debian stretch for use with selenium and, while the install goes fine, it crashes on boot with a SIGTRAP I have tried installing about 10 different versions of Chrome and I get the same output:

-chrome --version

> trace trap google-chrome If I run with

I get the following output (truncated showing the last few lines) > mprotect(0x7f91db20e000, 4096, PROT_READ) = 0 mprotect(0x7f91dbaca000, 28672, PROT_READ) = 0 mprotect(0x7f91dbe13000, 20480, PROT_READ) = 0 mprotect(0x7f91db8a0000, 12288, PROT_READ) = 0 mprotect(0x560e58168000, 8253440, PROT_READ) = 0 mprotect(0x7f91dc9c7000, 4096, PROT_READ) = 0 munmap(0x7f91dc9ba000, 51848) = 0 set_tid_address(0x7f91dc90bed0) = 12969 set_robust_list(0x7f91dc90bee0, 24) = 0 rt_sigaction(SIGRTMIN, {sa_handler=0x7f91dc388bd0, sa_mask=[], sa_flags=SA_RESTORER|SA_SIGINFO, sa_restorer=0x7f91dc3940e0}, NULL, 8) = 0 rt_sigaction(SIGRT_1, {sa_handler=0x7f91dc388c60, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART|SA_SIGINFO, sa_restorer=0x7f91dc3940e0}, NULL, 8) = 0 rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0 getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0 getrandom("\31\366\336bs\314C\251", 8, 0) = 8 getrandom("\340\364w\342\"\321\363\304", 8, 0) = 8 mmap(0xb8400000000, 8589934592, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory) mmap(0xb8400000000, 8589934592, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory) --- SIGTRAP {si_signo=SIGTRAP, si_code=SI_KERNEL} --- +++ killed by SIGTRAP +++ Thank you!!

I have SSH to a server and have a syslog at usr/local. I am using the command tail -f syslog.log | grep fps and want to save the output of it to a file. I have tried to SCP as below % scp root@10.211.128.73:/usr/local/syslog.log ~/Desktop/a-file.txt but got as -sh: /root: Is a directory Can anyone please help me out? additionally my ultimate aim if to read the FPS values from the file through selenium. Or is it possible to read the FPS values directly after the grep command. any suggestion would be great. Many thanks in advance.

I'm running selenium on linux (ubuntu) but I beleive it is an old version. Can someone tell me whether there's an easy way to update selenium on ubuntu using command line? I couldn't find anything online explaining how to do this? Thanks.

I have a script that I want to run on Linux server That script downloads CSV file from amazon after log in. I have achieved the goal with **Chrome web driver (Chromium) using selenium in Python**. My question is that Can I run this script on Linux **server without headless mode**? because when I run script in headless mode the file doesn't download.

I’m trying to do something complicated and I need some help. I’m able to record a session (video and audio) with python using xvfb and ffmpeg. I just want to visit a page using selenium and record the video and audio. It’s working but when I try to run multiple instances, the audio is mixing between records. How can achieve that? I’m using pulseaudio. I know that with alsa you can set like a interface per instance, but I don’t know how to do it. Thanks for you support and comments.

I am trying to execute headless firefox on the remote machine(running Ubuntu 16.04) through Selenium via SSH. However, this gives me a "Error: GDK_BACKEND does not match available displays" error. My host machine runs Windows. I do not want to see the graphical output. It is just being used for selenium testing. I am using X Virtual Frame Buffer to act as a dummy driver: Xvfb :10 -screen 0 1024x768x16 & I also have exported the DISPLAY environment variable with the value of 10 for this specific case. Where am I going wrong? EDIT: When I simply run "sudo firefox" in my commandline over SSH after running xvbf, no errors are thrown. Errors are only thrown when running firefox through selenium. **More Details:-** I am calling firefox through selenium. The exact error that the selenium standalone server gives is:- 17:52:55.218 INFO - Executing: [new session: Capabilities [{browserName=firefox, platform=ANY, firefox_profile=UEsDBBQAAAAAAJuOD0nf9RXUMgAAA...}]]) 17:52:55.230 INFO - Creating a new session for Capabilities [{browserName=firefox, platform=ANY, firefox_profile=UEsDBBQAAAAAAJuOD0nf9RXUMgAAA...}] org.openqa.selenium.firefox.NotConnectedException: Unable to connect to host 127.0.0.1 on port 7055 after 45000 ms. Firefox console output: Error: GDK_BACKEND does not match available displays

I'm using selenium to scrap a website on a Linux server with no display(Xvfb) running Debian 9 OS(Google Compute Engine). But, the website returns following body: name of website It seems that your browser is using traffic compression, please, turn it off in browser settings or use another one. I don't exactly understand what's browser's traffic compression. But, I've tried following things: 1. used latest version of Firefox and Chrome 2. Set accept-encoding of firefox webdriver to empty string. FirefoxProfile firefoxProfile = new FirefoxProfile(); firefoxProfile.setPreference("browser.privatebrowsing.autostart",true); firefoxProfile.setPreference("network.http.accept-encoding", ""); FirefoxOptions firefoxOptions = new FirefoxOptions(); firefoxOptions.setProfile(firefoxProfile); driver = new FirefoxDriver(firefoxOptions); Other websites can be browsed properly, and the same website works fine on my own Windows 10 workstation. But, it returns above body in linux.

I tried to install Ruby selenium-webdriver but I got this error: sudo gem install selenium-webdriver [sudo] password for sam: Building native extensions. This could take a while... ERROR: Error installing selenium-webdriver: ERROR: Failed to build gem native extension. current directory: /var/lib/gems/2.4.0/gems/ffi-1.9.18/ext/ffi_c /usr/bin/ruby2.4 -r ./siteconf20170509-10458-cbn64m.rb extconf.rb mkmf.rb can't find header files for ruby at /usr/lib/ruby/include/ruby.h extconf failed, exit code 1 Gem files will remain installed in /var/lib/gems/2.4.0/gems/ffi-1.9.18 for inspection. Results logged to /var/lib/gems/2.4.0/extensions/x86_64-linux/2.4.0/ffi-1.9.18/gem_make.out I've tried different methods that I found from different forums but they didn't work for me.Ruby version is 2.4 and Ubuntu version is 16.04

I'm writing a webpage-monitoring program that, every hour, logs in to a website using Selenium and notifies me if the page has changed in a particular way. For example, this program can monitor my cell phone data usage and warn me if my usage is spiking. However, I'm concerned about having my plaintext password just sitting on a machine that's perpetually connected to the Internet. **Q: What is a responsible way to automate periodic website logins?** I've considered the following precautions, in order of increasing paranoid-ness: 1. "Harden" the Linux distro -- turn off ping responses, CUPS, Java, SSH, Samba, and other unused services. 2. Encrypt the password on disk and hard-code the key in the program (like, use XOR to obfuscate the password a little). 3. Store the password only in RAM; that is, enter the password when I start the program. (So the program is always running; can't use cron) 4. Turn off the machine's network when the program isn't running using ifdown and ifup. 5. Use a timed electrical outlet to -- once an hour -- physically power up the machine, let it run for a few minutes, and power it down (say, using this [Art Controller relay board](http://shop.evilmadscientist.com/productsmenu/tinykitlist/580)) . (BTW, I have the basic program working on a Raspberry Pi using Python and pyvirtualdisplay as described here , and I'm using Selenium because some websites of interest require JavaScript.)

I'm firing up Selenium server with a bash script and as you can see from the timestamps on the log below, it takes about 32 seconds for the thing to fully come online: Feb 28, 2012 10:19:02 PM org.openqa.grid.selenium.GridLauncher main INFO: Launching a standalone server 22:19:02.835 INFO - Java: Sun Microsystems Inc. 20.0-b11 22:19:02.836 INFO - OS: Linux 2.6.32-220.4.1.el6.x86_64 amd64 22:19:02.852 INFO - v2.19.0, with Core v2.19.0. Built from revision 15849 22:19:02.988 INFO - RemoteWebDriver instances should connect to: http://127.0.0.1:4444/wd/hub 22:19:02.990 INFO - Version Jetty/5.1.x 22:19:02.992 INFO - Started HttpContext[/selenium-server/driver,/selenium-server/driver] 22:19:02.993 INFO - Started HttpContext[/selenium-server,/selenium-server] 22:19:02.993 INFO - Started HttpContext[/,/] 22:19:34.552 INFO - Started org.openqa.jetty.jetty.servlet.ServletHandler@488e32e7 22:19:34.552 INFO - Started HttpContext[/wd,/wd] 22:19:34.555 INFO - Started SocketListener on 0.0.0.0:4444 22:19:34.555 INFO - Started org.openqa.jetty.jetty.Server@7d29f3b5 Instead of using a "sleep 32" command after starting the server (to delay the script before moving on), I'd like my bash script to wait until it sees the string "Started SocketListener", and then continue. Is that possible?