Cookbooks
Dart
Chapter 11. HTML and the Web
Downloading a web page

Dart:
Downloading a web page

How to:

Dart provides the http package, a popular third-party library for making HTTP requests. Here’s a basic example of how to use it to download a webpage:

First, add the http package to your pubspec.yaml:

dependencies:
  http: ^0.13.3

Then, import the package and use it to fetch the content of a web page:

import 'package:http/http.dart' as http;

Future<void> main() async {
  var url = Uri.parse('http://example.com');
  var response = await http.get(url);
  if (response.statusCode == 200) {
    print('Page downloaded:');
    print(response.body);
  } else {
    print('Request failed with status: ${response.statusCode}.');
  }
}

Sample output (this will vary based on the content of the web page):

Page downloaded:
<!doctype html>
<html>
<head>
    <title>Example Domain</title>
...
</html>

For more complex scenarios, like handling cookies or setting user-agent headers, you would use the same http package but with additional configurations to your request:

import 'package:http/http.dart' as http;

Future<void> main() async {
  var headers = {
    'User-Agent': 'YourCustomUserAgent/1.0',
    'Cookie': 'name=value; name2=value2',
  };
  var url = Uri.parse('http://example.com');
  var response = await http.get(url, headers: headers);

  if (response.statusCode == 200) {
    print('Page downloaded with custom headers:');
    print(response.body);
  } else {
    print('Request failed with status: ${response.statusCode}.');
  }
}

Using headers like these can mimic browser requests more accurately, which is particularly useful when dealing with sites that have specific requirements or protections against scraping.

Parsing HTML