Configure several schedules for different jobs
Automation is key when it comes to web data acquisition. If you have identical scraping jobs that you execute on a regular basis, let Scheduler do the work for you. Create multiple jobs, use the Cron expression to set the time intervals, and get automatic data updates at ease.
Get data automatically delivered to your cloud storage
Enjoy peace of mind knowing that with scheduled jobs, you’ll receive updates and data to your preferred cloud storage automatically and regularly.
Receive notifications once the data is transferred
To receive regular data updates, you’ll no longer need to send new requests and create or maintain scheduling scripts. Just set up recurring scraping jobs, and we’ll do the rest – once the scraped and parsed data is sent to your cloud storage, you’ll receive a notification.
How does Scheduler work?
Scheduler is a feature of all our Scraper APIs for setting up recurring scraping and parsing jobs. Its process can be broken down into three parts: you set up the frequency using Cron, provide the scraping and parsing parameters, and set the end time.
Step 1
Set a frequency rate at which we’ll be executing the scraping jobs.
Step 2
Provide us with the job parameter sets of your choice.
Step 3
Get a notification as soon as the result is in your cloud storage.
Step 4
Set the end time and we’ll stop delivering the data.
Scraper APIs with the Scheduler feature
Scheduler is an additional feature any of our Scraper API users get for free.
SERP Scraper API
City-level SERP data delivery from major search engines.
Localized search results
Live and reliable data
Resilient to SERP layout changes
Best for:
Keywords data collection, brand monitoring, ads data tracking.
From $49/month
E-Commerce Scraper API
E-commerce product page scraping with ready-to-use data.
1000s of e-commerce websites
Adaptive Parser
Structured data in JSON
Best for:
Pricing intelligence, product catalog mapping, competitor analysis.
From $49/month
Web Scraper API
Scalable real-time data collection from a majority of websites.
Customizable request parameters
JavaScript rendering
Convenient delivery
Best for:
Website changes monitoring, fraud protection, travel fare monitoring.
From $49/month
Real Estate Scraper API*
Real-time property data gathering from popular real estate websites.
Zillow, Redfin, and others
No CAPTCHAs or IP blocks
Cost-efficient scraping
Best for:
Researching new investments, price optimization, trend discovery.
*Part of Web Scraper API
Aivaras Steponavicius
Senior Account Manager @ Oxylabs
By scraping and parsing data according to a set interval, Scheduler does the heavy lifting – the data is sent to the client’s cloud storage so they don’t need to send identical requests repeatedly.
Ruta Petronyte
Senior Account Manager @ Oxylabs
Scheduler is there to simplify the scraping and parsing jobs you do routinely. According to your preference, it can execute scraping and parsing jobs every minute, hour, or day.
A word from your Dedicated Account Manager
Get personal assistance from a Dedicated Account Manager regarding any web scraping-related matter.
Useful resources
Step-By-Step Tutorial
Follow this detailed video guide to getting started with Scheduler.
Where to Start & How it Works
Read this article to learn more about automating your web scraping and parsing jobs.
In-depth Tutorial
Check this detailed tutorial with code examples on how to use Scheduler.
Frequently Asked Questions
What is Scheduler?
Scheduler is a free feature of Oxylabs’ Scraper APIs. It allows you to automate regular and identical scraping jobs as well as receive data at the frequency of your choice. This way, you no longer need to send new requests or create scheduling scripts to create repetitive scraping and parsing jobs.
How to set up jobs with Scheduler?
You have to submit a request for us to create a schedule, where you specify:
- How often should we repeat the job(s)
- What exactly is the job
- Job end time
- Callback URL (optional)
- Cloud storage URL (optional)
If you provide us with the callback URL, we’ll send you a notification once the scheduled job is done. You’ll be able to access the scraped and parsed results in your cloud storage (if you specified it.)
Get the latest news from data gathering world
Scale up your business with Oxylabs®
GET IN TOUCH
General:
hello@oxylabs.ioSupport:
support@oxylabs.ioCareer:
career@oxylabs.io
Certified data centers and upstream providers
Connect with us
Advanced proxy solutions
Resources
Innovation hub