Introduction

Remilia is a high-performance web scraping framework designed for efficiency.

Overview

func main() {
	// Create a new instance of remilia
	rem, err := remilia.New()
	if err != nil {
		fmt.Println("Error creating new remilia instance:", err)
		return
	}
 
	// Define the titleParser as a function to parse and print titles from a web page
	titleParser := func(doc *goquery.Document, put remilia.Put[string]) {
		doc.Find("h1").Each(func(_ int, s *goquery.Selection) {
			fmt.Println(s.Text()) // Print each <h1> tag's text
		})
	}
 
	// Execute the web scraping task using the remilia instance
	err = rem.Do(
		rem.Just("https://go.dev/"), // The URL to fetch
		rem.Unit(titleParser),       // The parser function to use
	)
	if err != nil {
		fmt.Println("Error during remilia task:", err)
	}
}
 

Features

  • Clean API & elegant mental model
  • Concurrency supporting
  • Configurable backoff retry algorithm
  • Pre-request & post-response hooks supporting