Before you read the result
What Kaspersky VPN documents about these signals
Each point below is drawn from Kaspersky's own knowledge base article on the product and its online help. Together they cover most of what the checks on this page can return.
The app starting is not the tunnel starting
Kaspersky documents that by default the application, and not the traffic encryption itself, is automatically launched when the operating system starts. The app then watches for situations where encryption could be useful and offers to switch it on. Traffic encryption only starts with explicit user consent. A check returning your own address is consistent with the app running while protection was never enabled, which is the first thing to rule out.
The exit servers belong to a partner
Kaspersky states that the servers used by the product sit in regions including the USA, Germany and Singapore, and that they are provided by its partner, the software company Pango. A lookup on the network behind your exit address will therefore name that infrastructure rather than Kaspersky itself. Knowing this beforehand stops an unfamiliar network name reading as evidence that something went wrong.
You can hand it your own resolver
The paid version carries a setting called Use custom DNS over HTTPS server, and Kaspersky notes that to set one up you need to turn off the VPN first. If you have ever used it, the resolver named on the DNS line is the one you configured rather than a default of the product. A name you do not recognise on that line may simply be an entry you made and forgot.
Kill Switch ships turned off
Kaspersky documents Kill Switch as blocking your Internet access if the secure connection is interrupted, and states plainly that by default the feature is turned off and the application does not block Internet access when that happens. It is reached through the application settings. Confirm its state before reading anything into a single result.
Split tunneling rules follow file paths
Kaspersky's split tunneling settings hold two lists, Do not use VPN and Use VPN, each populated through a Select button. Its documentation adds a caveat worth knowing: the settings for selected applications are applied to the specified paths only, so if a path changes because an application was updated, the settings will not be automatically applied to it. A browser that was excluded, or has quietly stopped being included, changes every reading below.