Go Scrapper API

As I was exploring further about Go, I came across APIs in Go, which was very easy to build. So instead of going the traditional way and creating a CRUD API, I wrote a simple scrapper API. The API scraps Goodreads quotes page and returns a JSON of quotes with their authors.

Here I am using 2 modules,

• Gin — For quickly building the API
• Colly — For web scrapping.

I have built it and deployed it on Heroku.

Commands I ran from start to finish

To initiate the current folder/project as a module

go mod init vigneshm.me/go-quotes-api

This creates a file called go.mod. This has the name of the project and the version of Go it uses.

To import modules

go get github.com/gin-gonic/gin
go get github.com/gocolly/colly

This imports the modules and creates a go.sum file to add checksums for the modules it downloads. Also, the go.mod page gets filled with all the dependent modules info.

Next, I filled in the main.go with my code.

Now, for the part where I deploy the app in Heroku.

Install Heroku CLI

npm install -g heroku

Login to Heroku

heroku login -i

Initiate a Git repo and commit the code

git init
git add .
git commit -m "initial commit"

Create a Procfile for Heroku

touch Procfile
echo "web: bin/go-quotes-api" > Procfile

This tells Heroku to run the command on startup.

Create Heroku App

heroku create go-quote-api
git push heroku master

This creates the Heroku app and pushes the code.

Heroku takes this code runs the build and exposes the app at https://app-name.herokuapp.com/

My app is available at https://go-quote-api.herokuapp.com/quotes/dumbledore

The last word can be replaced with any search criteria to search for quotes from your favorite character or book. Code can be found at https://github.com/vigneshm243/go-quotes-api.

Happy Coding!

Leave a 👏 if you liked the article, comment, and let me know what you think.

Originally published at https://vigneshm.me/posts/go_api/.