Performance benchmarks comprise an important component of this course. We draw examples from academic research and industry studies. This assignment involves the construction of a performance benchmark across popular Go backend frameworks.

Management Problem #

Managers of a technology startup are evaluating alternative software stacks for the company’s web applications. They know that Go will serve the company’s needs for backend web servers and applications, but they are uncertain about which backend web framework to use, if any.

Searches at https://go.dev/ point to numerous backend web frameworks, providing utilities that complement the Go standard library, including routing, session management (with cookies), security, and system logging. As well, there is a contingent of engineers arguing that the Go standard library is sufficient and that no Go web framework be used for application development.

Echo is described as a minimalist but extensible framework. Fiber, which draws on fastHTTP rather than net/http from the Go standard library, is known for its fast performance. And Gin is the most popular Go web framework. Golang Dojo offers a playlist of introductory videos on Go Web App development.

Scalability and performance are key in selecting a web application development environment. The firm’s data scientists suggest that comprehensive performance benchmarks be run across four web frameworks being considered: None, Echo, Fiber, and Gin.

Lessons from Industry Benchmarks #

TechEmpower runs performance benchmarks across hundreds of backend web frameworks, reporting results online, and documenting research methods in a public GitHub repository. Results across Go backend frameworks show great variability both in throughput and latency.

TechEmpower’s throughput test is called Fortunes:

In this test, the framework’s ORM is used to fetch all rows from a database table containing an unknown number of Unix fortune cookie messages (the table has 12 rows, but the code cannot have foreknowledge of the table’s size). An additional fortune cookie message is inserted into the list at runtime and then the list is sorted by the message text. Finally, the list is delivered to the client using a server-side HTML template. The message text must be considered untrusted and properly escaped and the UTF-8 fortune messages must be rendered properly.

Here are the throughput results across Go web frameworks:

TechEmpower uses a single-query latency test:

In this test, each request is processed by fetching a single row from a simple database table. That row is then serialized as a JSON response.

Here are TechEmpower’s latency results across Go web frameworks:

Assignment Requirements #

Assume the role of a company data scientist. Design and conduct a benchmark study across the four web framework options: None, Echo, Fiber, and Gin.

Examine both throughput and latency in querying an SQLite database, creating at least two queries for both throughput and latency. Define appropriate response measures for throughput and latency.

For database queries, use the Go package sqlite with its GitLab repository, which provides a Go port of the C-language database SQLite. It is not necessary to use an object-relational mapper for this study. But if an object-relational mapper is used, ensure that it is used across each of the four web framework options.

Conduct a Monte Carlo performance benchmark, running each query task at least one hundred times, obtaining response distributions across the runs, and computing average response measures across the runs for each treatment and each task. Construct tables and figures summarizing study results.

The README.md file of the public GitHub repository for this assignment should provide

• A description of repository folders and files
• Information about the research design (treatment conditions and data)
• Instructions about how to run the benchmark study
• Tables and figures summarizing benchmark results
• Recommendation for management: Based on performance indices alone, which web framework should the firm use?

Utilize test-driven development. The testing package in the Go standard library provides methods for testing and benchmarking. And the go test tool is bundled into the Go programming environment. Bates and LaNou (2023) and Bodner (2021) provide Go programming examples of testing and benchmarking as needed for this assignment. Check out Chris James’s GitBook Learn Go with Tests. Chloé Powell provides a brief introduction to unit testing in Go, highlighting the testing and testify packages: Unit Testing in Golang. Adelina Simion (2023) provides a comprehensive review of testing in Go. Want to take your unit testing to the next level? Check out the GitHub repository for testify.

Grading Guidelines (100 Total Points) #

• Coding rules, organization, and aesthetics (20 points). Effective use of Go modules and idiomatic Go. Code should be readable, easy to understand. Variable and function names should be meaningful, specific rather than abstract. They should not be too long or too short. Avoid useless temporary variables and intermediate results. Code blocks and line breaks should be clean and consistent. Break large code blocks into smaller blocks that accomplish one task at a time. Utilize readable and easy-to-follow control flow (if/else blocks and for loops). Distribute the not rather than the switch (and/or) in complex Boolean expressions. Programs should be self-documenting, with comments explaining the logic behind the code (McConnell 2004, 777–817).
• Testing and software metrics (20 points). Employ unit tests of critical components, generating synthetic test data when appropriate. Generate program logs and profiles when appropriate. Monitor memory and processing requirements of code components and the entire program. If noted in the requirements definition, conduct a Monte Carlo performance benchmark.
• Design and development (20 points). Employ a clean, efficient, and easy-to-understand design that meets all aspects of the requirements definition and serves the use case. When possible, develop general-purpose code modules that can be reused in other programming projects.
• Documentation (20 points). Effective use of Git/GitHub, including a README.md Markdown file for each repository, noting the roles of programs and data and explaining how to test and use the application.
• Application (20 points). Provide instructions for creating an executable load module or application. The application should run to completion without issues. If user input is required, the application should check for valid/usable input and should provide appropriate explanation to users who provide incorrect input. The application should employ clean design for the user experience and user interface (UX/UI).

Assignment Deliverable #

• Text showing the link (URL) for the public GitHub repository for the assignment

References #

• Bates, Mark, and Cory LaNou. 2023. Go Fundamentals: Gopher Guides. Boston: Addison-Wesley. [ISBN-13: 978-0-13-791830-0] Chapter 7, Testing, pages 195–229.
• Bodner, Jon. 2021. Learning Go: An Idiomatic Approach to Real-World Go Programming. Sebastopol, CA: O’Reilly. [ISBN-13: 978-1-432-07721-3] Book website at https://learning-go-book.dev/. GitHub code repository at https://github.com/learning-go-book. Chapter 13, Writing Tests, pages 271–297.
• Huson, J. Ben. 2018. Echo Quick Start Guide: Build Lightweight and High-performance Web Apps with Echo. Birmingham, UK: Packt. [ISBN-13: 978-1789139433] Code available from the book’s GitHub repository.
• James, Chris. 2023. Learn Go with Tests. GitBook available online at https://quii.gitbook.io/learn-go-with-tests/
• Labouardy, Mohamed. 2021. Building Distributed Applications in Gin: A Hands-On Guide for Go Developers to Build and Deploy Distributed Web Apps with the Gin Framework. Birmingham, UK: Packt. [ISBN-13: 978-1801074858] Code available from the book’s GitHub repository.
• McConnell, Steve. 2004. Code Complete: A Practical Handbook of Software Construction (second edition). Redmond, WA: Microsoft Press [ISBN-13: 978-0-7356-1967-8] Chapter 32, Self-Documenting Code, pages 777–817.
• Simion, Adelina. 2023. Test-Driven Development in Go: A Practical Guide to Writing Idiomatic and Efficient Go Tests Through Real-World Examples. Birmingham, UK: Packt. [ISBN-13: 978-1803247878]

Back to the Analytics Systems Engineering main page.