IMPLEMENTATION of DSL for WEB SCRAPING

Abstract

The main goal of this project is to implement a DSL for Web Scraping. A Domain Specific Language or DSL in short is a language that is created for solving a single purpose. It is a language that is used in only one domain. In our project, that domain is web scraping. Our main aim is to create a simple scripting language with easy to use syntax with many features that help the user scrape the web easily. Currently, web scraping is a tedious process. At the moment, the majority of web scraping is done by the means of modules in high level languages. This would require the user and in-depth knowledge of the high-level language as well, and thus precludes many laymen from easy web scraping. This project will provide a DSL with highly simplified syntax which does not assume any skill from the user. Thus, anyone would be able to use this language to scrape the web with no previous knowledge of the domain. This DSL has been implemented using Python and its scraping libraries. With this, many features and functionalities can be implemented in the DSL thus providing an effective tool for web scraping without compromising on simplicity Keywords— Domain Specific Language, Web Scraping, Python, Beautiful Soup 4