The next question was, of course, what was happening and why was it possible? Connecting to an AP is done in two phases. First, the client and AP agree on the connection parameters and what encryption to use. Second, they perform a so-called WPA handshake or four-way handshake, where they exchange the encryption parameters and make sure they both have the PSK. After this, they open an encrypted data connection. But the test case I was running skipped the WPA handshake, so something had to be happening before the handshake, where the AP and client agreed on the connection parameters and what encryption to use.
The first phase consists of a probe request and response, an authentication request and response, and finally an association request and response. The main purpose of the probe request is just to discover the AP. The probe response then contains all the information about the AP, including which encryptions are supported. The next request-response pair is authentication, the purpose of which is to ensure backward compatibility. The next thing is the association request and response. In the association request, the client tells the AP which encryption and what parameters it wants to use. The association request also opens the data connection between the AP and client.
The test case I was running had a normal probe request and authentication request. Then the association request claimed that the client supported WPA1 encryption. At this point, the WPA handshake is supposed to happen, but since my test case skipped the handshake, the AP and client started to send data frames without any encryption.