Downloading a web page

Kotlin:
Downloading a web page

How to:

Let’s roll with Kotlin’s HttpURLConnection to quickly nab a web page. We’ll also use coroutines for smooth background ops. Here’s a primer:

import java.net.HttpURLConnection
import java.net.URL
import kotlinx.coroutines.*

fun main() = runBlocking {
    val url = "http://example.com"
    val result = withContext(Dispatchers.IO) {
        downloadWebPage(url)
    }
    println(result)
}

fun downloadWebPage(urlAddress: String): String {
    val url = URL(urlAddress)
    val connection = url.openConnection() as HttpURLConnection
    try {
        connection.connect()
        return connection.inputStream.bufferedReader().use { it.readText() }
    } finally {
        connection.disconnect()
    }
}

Sample output:

<!doctype html>
<html>
<head>
    <title>Example Domain</title>
...
</html>

Nice, eh? You’ve got the web page’s HTML!

Deep Dive

Downloading web pages is as old as the web itself. In the ’90s, folks used command-line tools like wget and curl. They’re still around but when you want more control or to integrate web content fetching into an app, you code it.

In Kotlin, you could use Java’s HttpURLConnection or libraries like OkHttp or Ktor for a powerful approach with more features. The example above is bare-bones; in real life, you’d think about error handling, redirects, and performance. Maybe add in retries or a timeout? And you can’t forget about handling different character encodings and content types.

You’d also consider threading. Wouldn’t wanna hang the main thread while fetching a giant page, would we? Hence, coroutines - they let your app stay responsive, fetching in the background without breaking a sweat.

Kotlin:Downloading a web page

How to:

Deep Dive

See Also

Kotlin:
Downloading a web page