Testing

Testing is an essential part of software development that ensures your code works as expected and helps maintain its reliability over time.

Pytest is a powerful, no-boilerplate-needed testing framework in Python that makes writing simple and scalable test cases easy. Let’s explore how you can leverage Pytest in your Python projects.

Why Pytest?

Pytest is a popular testing framework for Python due to its simplicity and flexibility. It supports powerful fixtures, has a rich plugin architecture, and can easily integrate with other testing tools and frameworks.

With Pytest, writing tests becomes more pythonic and less cumbersome compared to other frameworks like unittest.

Setting Up Pytest

First, you need to install Pytest. Simply run:

pip install pytest

After installation, you can start writing your test cases.

Example Project Structure

Imagine you have a simple Python project with the following structure:

• my_project/
  • app.py
  • tests/
    • test_app.py

In app.py, you have a function you want to test. For instance:

# app.py

def add_numbers(a, b):
    """Add two numbers."""
    return a + b

Writing Your First Test

Now, let’s write a test for the add_numbers function in test_app.py.

# tests/test_app.py

from app import add_numbers

def test_add_numbers():
    assert add_numbers(2, 3) == 5

This test checks if the add_numbers function correctly adds two numbers.

Running the Tests

To run your tests, simply execute:

pytest

Pytest will automatically discover and run all tests in the tests directory.

Parametrizing Tests

One of the powerful features of Pytest is parameterized tests. This allows you to run the same test function with different inputs. For example:

# tests/test_app.py

import pytest
from app import add_numbers

@pytest.mark.parametrize("a, b, expected", [(2, 3, 5), (4, 5, 9), (0, 0, 0)])
def test_add_numbers_parametrized(a, b, expected):
    assert add_numbers(a, b) == expected

This will run test_add_numbers_parametrized three times with different sets of arguments.

Handling Exceptions

Testing exceptions is straightforward with Pytest. Suppose you have a function that raises an exception:

# app.py

def divide_numbers(a, b):
    if b == 0:
        raise ValueError("Cannot divide by zero")
    return a / b

You can write a test to ensure the exception is raised correctly:

# tests/test_app.py

import pytest
from app import divide_numbers

def test_divide_numbers_exception():
    with pytest.raises(ValueError):
        divide_numbers(10, 0)

Conclusion

Testing with Pytest is a vital part of the Python development process. It’s straightforward, flexible, and powerful. By integrating testing into your workflow, you can write more reliable and maintainable code. Remember, a well-tested application is a robust application!

Other Reading

4 Techniques for Testing Python Command-Line (CLI) Apps – Real Python

Get Started — pytest documentation

Pytest – Starting With Basic Test

Testing Python Applications with Pytest – Semaphore Tutorial

Some good general links:

https://www.youtube.com/channel/UCWHJGQc7Vqo37dlUmpiK6hQ/videos

https://www.samueltaylor.org/articles/speed-up-web-scraping.html

https://www.adamsmith.haus/python/examples/4419/beautifulsoup-find-an-element-by-its-id

https://datasetsearch.research.google.com/

https://ianlondon.github.io/blog/web-scraping-discovering-hidden-apis/

What is Agile?

Agile project management is an approach to software development and project management that prioritizes flexibility, collaboration, and customer-centricity.

It involves iterative progress and adaptability to change, often breaking down larger projects into smaller, manageable parts known as iterations or “sprints.”

Agile methods are not confined to software development; they can be applied to other fields such as marketing, manufacturing, and organizational change.

Agile in the Tech World

Using Agile methodologies in for example website design allows for a more flexible, adaptive, and customer-centric approach.

Rather than spending months perfecting every aspect of the site before launch, teams can break the project down into manageable “sprints” or “iterations.”

This enables them to release a Minimum Viable Product (MVP) more quickly and then iteratively improve upon it based on real-world feedback.

In this way, design and functionality can evolve in response to user needs and changing market conditions, leading to a more effective and successful end product.

Agile in diagrams

you want to avoid this:

Useful Terminology

Iteration= A time-boxed period during which a team completes a defined set of tasks. Also known as a “sprint” in Scrum methodology.

User Stories= Short, simple descriptions of a feature or function written from the perspective of an end user.

Backlog= A prioritized list of tasks, features, bug fixes, and technical work that needs to be done in a project.

Scrum Master= The individual responsible for ensuring that the Scrum process is understood and followed by the team.

Product Owner= The person responsible for defining, prioritizing, and updating the product backlog to deliver maximum value to the business.

Sprint= A specific type of iteration in Scrum, usually lasting between one to four weeks.

Daily Stand-up= A brief daily meeting where team members discuss what they worked on the previous day, what they will work on today, and any blockers they might have.

Kanban Board= A visual tool used to manage workflow and optimize efficiency by tracking tasks and their statuses.

Swimlanes= Columns or rows in a Kanban board that categorize different types of work or differentiate between different teams.

Epic= A large user story that is too complex to be completed in a single iteration and must be broken down into smaller user stories.

Velocity= A metric used to measure the amount of work a team can accomplish during a single iteration.

Burndown Chart= A graphical representation of work left to do versus time, often used in Scrum to track the progress of a sprint.

Retrospective= A meeting held at the end of each iteration to discuss what went well, what needs improvement, and how to implement those improvements.

Expansion on terms

Burn down chart

as its’ name implies a burndown chart is a chart that shows the burndown of work.

So basically you see the number of story points

the team is delivering over time, so your team’s velocity.

On the X axis you have your sprints and on your Y axis, your story points

Retrospective

What went well?

• Great teamwork and collaboration.
• Effective communication.
• The team worked in parallel on multiple tasks and finished all of them.
• Management authorized additional resources for the project.
• Good collaboration with other teams.

What didn’t?

• Several unplanned issues came up in UAT.
• Testing took longer than anticipated.
• BAU (Business As Usual) got in the way of project work.

What could we do differently?

• Automate testing to reduce cycle time.
• Define additional resources for the project or handover BAU (if possible).

MVP

MVP (Minimum Viable Product), the Core of the Agile Methodology. An MVP is a concept from agile scrum that refers to a product that has just enough features to satisfy the needs of early customers and, more importantly, give them something to provide feedback on to shape the future of the product.

FAQ and Useful Info

What are the 3 core roles in an Agile team?

1. product owner typically an end customer who defines what is valuable and priroties
2. scrum master – runs team meetings, removes impedimentswork with product owner and ensures the team reamins on track
3. delivery team – people in charge of execution, defines which user stories are part of a sprint

What are good Agile Rituals?

• sprint planning
• daily standup
• sprint review (demo/showcase)
• retrospectives

Top Tips

• start with sprint 0 for planning
• define who is in the team and roles
• agree days and time for rituals
• make sure you have done due dillegence before you start
• understand constraints
• define when releases are done

Does Agile equate to Scrum? Are they equal or the same thing?

Technically no, in reality yes. Allow me to clarify this, Scrum is one of many Agile methodologies, but Scrum is the most popular and widely used of all Agile methodologies. Hence, when people are talking about “Agile” they are generally referring to Scrum. Most people don’t even know there are other methodologies besides Scrum. If you want to learn more about the other methodologies, just go to this link.

Explain agile in an interview

The Agile Way! Keep it simple and succinct. Just let the interviewer know that Agile is an iterative approach to managing projects. Highlight that you love Agile because it follows a customer-centric approach whilst closely collaborating with your team members to deliver value for the business in a short period of time. You can also mention a couple of the things found on this article.

Case Study

Breaking a complex project into manageable tasks

Breaking down a complex project into manageable tasks is fundamental in Agile methodologies. Here’s a simplified step-by-step process:

1. Define the Vision:

Start with the end goal or the main objective of the project. What are you trying to achieve? This can be documented in a vision statement or a product roadmap.

2 Create Epics:

An epic is a large chunk of work that can be broken down into smaller tasks. It’s more granular than the vision but not as detailed as individual tasks. For example, “User Authentication” might be an epic for a software project.

3 Develop User Stories:

For each epic, create user stories. A user story is a description of a feature or requirement from an end-user perspective. It typically follows the format: “As a [type of user], I want [an action] so that [a benefit/a value]”.

4 Break Down User Stories into Tasks:

A user story can often be broken down into multiple tasks. For instance, for the user story “As a user, I want to reset my password”, tasks could be:
– Design the “Forgot Password” interface.
– Develop backend logic for password reset.
– Implement email service for sending reset links.
– Test the password reset functionality.

5 Estimate Tasks:

Once tasks are identified, the team can estimate the time or effort required for each task. In Agile, this could be in story points or hours, depending on the team’s preference.

6 Prioritize:

Determine the order of importance for your user stories and tasks based on business needs, dependencies, risks, and other factors.

7 Plan Sprints:

In Scrum (a type of Agile methodology), work is divided into sprints, which are time-boxed iterations (often 2-4 weeks). Place your prioritized tasks into these sprints based on the team’s capacity and the project timeline.

8 Review and Adjust:

After each sprint, conduct a retrospective. Discuss what went well, what didn’t, and how processes can be improved. This feedback loop ensures that the project remains adaptable and the team continues to improve.

9 Maintain a Backlog:

Not all tasks or stories will fit into the immediate sprints. Keep a well-maintained and prioritized backlog of these items. Over time, some backlog items might become irrelevant or new ones might be added.

10. Continuous Communication: Foster an environment of open communication. Ensure that team members can discuss challenges, clarify doubts, and collaborate on solutions. This keeps everyone aligned and ensures that tasks are accurately defined and understood.

Remember, the goal of Agile is flexibility and adaptability. As you move through the project, it’s okay (and expected) that tasks, priorities, and even the broader scope may change. The key is to maintain communication and ensure that the team is always working towards the most valuable outcomes for the project.

Case Study -Example

Let’s take the hypothetical scenario of developing a new e-commerce platform:

Vision: Create a user-friendly e-commerce platform where users can buy handcrafted products from local artisans.

Epics:

1. User Authentication
2. Product Browsing
3. Shopping Cart Management
4. Checkout and Payment
5. User Reviews and Ratings

User Stories (for the ‘User Authentication’ Epic):

1. As a new visitor, I want to register for an account so I can make purchases.
2. As a registered user, I want to log into my account to view my order history.
3. As a user, I want to reset my password in case I forget it.

Tasks (for the ‘Reset password’ User Story):

1. Design the “Forgot Password” interface.
2. Develop backend logic for password reset.
3. Implement email service for sending reset links.
4. Test the password reset functionality.

Estimate Tasks:

1. Design interface – 8 hours
2. Backend logic – 5 hours
3. Email service implementation – 3 hours
4. Testing – 4 hours

Prioritize:

1. User registration and login (critical for user acquisition)
2. Product Browsing (core feature)
3. Checkout and Payment (necessary for monetization)
4. Shopping Cart Management
5. User Reviews and Ratings (can be introduced as an enhancement in version 2)

Sprint Planning:
– Sprint 1: Complete the design and development of user registration and login, start on product browsing.
– Sprint 2: Complete product browsing and start on shopping cart management.
… and so on.

Review and Adjust (after Sprint 1):
– Found that users want social media login options.
– Adjust upcoming sprints to integrate social media authentication.

Backlog Items:

1. Implement a recommendation system based on user’s browsing history.
2. Offer gift wrapping options during checkout.
3. Create a loyalty rewards program.

Throughout the project, the team meets daily for stand-ups to discuss progress, blockers, and next steps. They also engage with stakeholders frequently to ensure the platform is aligning with business goals and user needs.

After the launch of the e-commerce platform, iterative feedback is gathered from real users, and the team continues to enhance the platform based on this feedback, ensuring its success and adaptability in the market.

Now, let’s evolve the e-commerce platform example by focusing on a comprehensive database design and implementation for it.

Vision: Develop a robust and scalable database for the e-commerce platform to efficiently manage products, user data, transactions, and reviews.

Epics:

1. Database Design
2. User Data Management
3. Product Data Management
4. Transaction Data Management
5. Reviews and Ratings Data Management

User Stories (for the ‘Product Data Management’ Epic):

1. As an admin, I want to add new products to the database with all related data points to list them for sale.
2. As a system, I need to categorize products based on several attributes (e.g., type, artisan, price range) to aid in product searching and filtering.
3. As an admin, I want to update or delete product details to maintain accurate inventory and product information.

Tasks (for the ‘Add new products’ User Story):

1. Create a table ‘Products’ with columns: ProductID, ProductName, Description, Price, ArtisanID, CategoryID, StockCount, ImageURL, etc.
2. Develop an admin interface for entering product data.
3. Integrate data validation checks to ensure product data integrity.
4. Implement backend logic to insert product data into the ‘Products’ table.
5. Test the product addition functionality.

Estimate Tasks:

1. Create ‘Products’ table – 3 hours
2. Design admin interface – 10 hours
3. Data validation checks – 5 hours
4. Backend logic for data insertion – 8 hours
5. Testing – 5 hours

Prioritize:

1. User Data Management (to support user registration and login)
2. Product Data Management (to list items for sale)
3. Transaction Data Management (essential for sales)
4. Reviews and Ratings Data Management (for user feedback)
5. Database Design (constant across all other epics, foundational)

Sprint Planning:
– Sprint 1: Set up initial database schema, focus on user data management.
– Sprint 2: Focus on product data management and start transaction data management.
… and so on.

Review and Adjust (after Sprint 1):
– Found that users also want to store multiple shipping addresses.
– Adjust upcoming sprints to add a ‘UserAddresses’ table in the database schema.

Backlog Items:

1. Implement database archiving for older transactions.
2. Design data redundancy and backup protocols.
3. Analyze database performance and optimize queries.

Throughout the project, the database team collaborates with frontend and backend teams to ensure seamless data flow. Regular backups, testing, and performance checks are scheduled to maintain data integrity and system health.

Once the database is up and running, periodic audits and optimizations are conducted based on system usage and growth patterns, ensuring its scalability and reliabil