Does the default Form Security Filter lock or block the transaction after a number of unsuccessful attempts? Is there any documentation which describes how it functions out of the box?
Would it be conceivably possible for a custom Form Security Filter to include similar functionality?
This question should read:
"How does the Form Submission Access Controller prevent brute force attacks on default resume form challenge response patterns"
... and "Can a Form Security Filter implementation add additional functionality to disable a transaction if too many attempts have been made on the challenge?"
There isn't a 'default' Form Security Filter. If you create one, it will run before the regular TM access checks, so it's an extra level of security.
So, it would be conceivably possibly to check for brute force attacks, but I would think that is better dealt with via wider system configuration rather than code.
What process runs when you don't create a security filter? Some default server side process checks that the challenge response matches the designated form data before serving up PII.
It's the Form Submission Access Controller service.
Comment on question to update specificity.