[FIXED] Error testing flask-socketio server emitted events with pytest

Issue

I'm building the test suite for a recently deployed webapp. It was built using flask-socketio and uses pytest for the testing suite.

The big issue here is the lack of documentation for tests with flask-socketio. I found some projects that include a testing suite: https://github.com/miguelgrinberg/Flask-SocketIO/blob/main/test_socketio.py https://github.com/miguelgrinberg/flack/blob/master/tests/tests.py

But none of those implement tests on server responses to messages.

In my case, I have the following listener in my server:

@socketio.on('getBettors', namespace='/coin')
def get_current_bettors_coin():
    """
    functionshould send current tickets to the client to build
    """
    game_name = "coinv1"
    coin_game = Game.objects(game_name=game_name)[0]
    current_time = get_utc_now_func()


    current_game = GameSeed.objects(
        game_id=coin_game.pk,
        start_time__lte=current_time,
        end_time__gt=current_time,
    )[0]
    
    current_game_tickets = GameTicket.objects(
        game_id=current_game.pk
    )

    list_bets = []
    for ticket in current_game_tickets:
        ticket_dict = {
            "user": User.objects(id=ticket.owner_id.pk)[0].username,
            "choice": str(ticket.user_choice),
            "value": str(ticket.bet_value),
        }
        list_bets.append(ticket_dict)
    emit('getBettors', {"bets": list_bets}, broadcast=True)

Basically the listener searches the DB to get all current bettors in a certain game (it's a betting game)

But I have no way to test if the result being emitted is the actual result to be expected, as there is no argument for such in the flask-socketio test client:

conftest.py

@pytest.fixture(scope='module')
def create_flask_app():
    #drop all records in testDatabase before strting new test module
    db = connect(host=os.environ["MONGODB_SETTINGS_TEST"], alias="testConnect")
    for collection in db["testDatabase"].list_collection_names():
        db["testDatabase"].drop_collection(collection)
    db.close()
    
    # Create a test client using the Flask application configured for testing
    flask_app = create_app()
    return flask_app

@pytest.fixture(scope='module')
def test_client(create_flask_app):
    """
    Establish a test client for use within each test module
    """
    with create_flask_app.test_client() as testing_client:
        # Establish an application context
        with create_flask_app.app_context():
            yield testing_client  # this is where the testing happens!

@pytest.fixture(scope='module')
def data_test_coin_toss_game():
    """
    Populate DB with mock data for coin toss game tests
    """
    #drop all records in testDatabase before strting new test class
    db = connect(host=os.environ["MONGODB_SETTINGS_TEST"], alias="data_test_coin_toss_game")

    mock_user_1 = User(
        username = "data_test_coin_toss_game",
        email = "[email protected]",
        password = "sdgibgsdgsdg",
        user_type = 1,
    )
    mock_user_1.create()

    first_deposit = Transaction(
        owner = mock_user_1,
        transaction_type = "deposit funds",
        currency_code = "MockCoin",
        value = Decimal("1000.50"),
        reference = "firstdeposit",
        datetime = datetime.datetime.utcnow(),
    )
    first_deposit.save()

    coin_game = Game(
        game_name = "coinv1",
        client_fixed_seed = hashlib.sha256("a random seed for the client".encode()).hexdigest(),
    )
    coin_game.save()

    base_time = get_utc_now_func()
    game_seed = GameSeed(
        game_id = coin_game,
        nonce = 4,
        seed_crypto = hashlib.sha256("wow a secret seed".encode()).hexdigest(),
        seed_client = hashlib.sha256("a random seed for the client".encode()).hexdigest(),
        result = Decimal("0"),
        start_time = base_time,
        lock_time = base_time + datetime.timedelta(days=1),
        reveal_time = base_time + datetime.timedelta(days=2),
        end_time = base_time + datetime.timedelta(days=3),
    )
    game_seed.save()

    db.close()

Test function:

def test_get_bettors_websocket(create_flask_app, test_client, data_test_coin_toss_game):
    """
    GIVEN a endpoint to retrieve all current bettors in a game
    WHEN a user communicates to that endpoint
    THEN check that the correct information is being sent out
    """
    client = socketio.test_client(create_flask_app)
    assert client.is_connected()

    received_return = False
    @client.on('getBettors', namespace='/coin')
    def process_request():
        print("AAA")
        global received_return
        received_return = True

    client.emit('getBettors', namespace='/coin')

    assert received_return

At this point I was just trying to figure out a way to check the message emitted from the server, without actually testing the business logic.

The test yields the following error:

FAILED tests/functional/test_coin_game.py::test_get_bettors_websocket - AttributeError: 'SocketIOTestClient' object has no attribute 'on'

Apparently the test client doesn't allow registering of "on" listeners. Does anyone know how to test such an application?

***I could theoretically encapsulate all the business logic of the server side listener in a different function, that could then be tested and just make the listener call the function. But that seems sloppy, error prone and, for other event listeners, the contents of the client message are used as arguments for queries, therefore I'd need to make the tests work in the way I proposed above

Solution

The Flask-SocketIO test client is not a real client, you cannot register event handlers. The test client records anything the server emits, and allows your test to check it. Flask-SocketIO's own unit tests do this here, for example.

        received = client.get_received()
        self.assertEqual(len(received), 3)
        self.assertEqual(received[0]['args'], 'connected')
        self.assertEqual(received[1]['args'], '{}')
        self.assertEqual(received[2]['args'], '{}')

Answered By - Miguel

Read More