Gerapy

screenshot of Gerapy
django
vue

Gerapy

Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js

Overview:

Gerapy is a distributed crawler management framework based on Scrapy, Scrapyd, Scrapyd-Client, Scrapyd-API, Django, and Vue.js. It allows users to create configurable projects, generate code for Scrapy, and manage the deployment and monitoring of jobs. Gerapy is developed using Python 3.x and can be installed via pip.

Features:

  • Distributed Crawler Management: Gerapy allows users to manage and distribute crawler tasks across multiple machines.
  • Configurable Projects: Users can create configurable projects and automatically generate Scrapy code.
  • Deployment and Monitoring: Gerapy provides tools for deploying projects and monitoring job progress.
  • Docker Support: Gerapy can be run in a Docker container for easy setup and deployment.
django
Django

Django is a high-level Python web framework that encourages rapid development and clean, pragmatic design. It follows the model-view-controller (MVC) architectural pattern, providing an extensive set of built-in tools and conventions to streamline the creation of robust and scalable web applications.

vue
Vue

Vue.js is a lightweight and flexible JavaScript framework that allows developers to easily build dynamic and reactive user interfaces. Its intuitive syntax, modular architecture, and focus on performance make it a popular choice for modern web development.

dashboard
Dashboard

A dashboard style website template is a pre-designed layout that features a user interface resembling a control panel or dashboard. It typically includes charts, graphs, tables, and other data visualization tools that allow users to monitor and analyze data in real-time.

docker
Docker

A website that uses Docker for containerization to streamline development, testing, and deployment workflows. This includes features such as containerization of dependencies, automated builds and deployments, and container orchestration to ensure scalability and availability.