8

Updated based on comments:

In what ways can we distinguish a human being doing certain activities online and a bot programmed to do similar activities, say checking email, downloading some music files, shopping on ebay, searching on Google etc., or maybe trying to deface/hack a website, brute force a log-in password etc.

To limit the scope of the question and make it more clear, let us restrict our observations only to network-oriented behavior, some examples being- the amount of time spent doing XYZ thing online, the amount/type of data downloaded (say) from a file sharing website, the number of friends/followers on Social media websites, etc.

I guess it should possible to obtain some 'patterns' which will distinguish human behavior and programmed behavior.

The Turing test is not what I am looking for.

What techniques can be useful here? Machine learning? Game theory?

References to relevant academic/research articles will also be good.

pnp
  • 763
  • 1
  • 7
  • 17

1 Answers1

10

The most common/obvious way is a challenge-response test that is easy for humans but hard for computers (of course, but not only, CAPTCHA).

This kind of test is very effective{1} but falls under the HIP (Human Interactive Proofs) area: it's not transparent.

Typical, "simple" approaches to distinguish human website traffic from Bot are:

  • time it takes to populate all the fields and click the submit button of an input form (frequently used but simple to bypass).

    Watching the cadence / pace of the communication is a more secure alternative (this is one of the feature of Google's No CAPTCHA reCAPTCHA);

  • honeypots (i.e. traps for bots that consist of a link or field present on the page that isn't visible to the human eye)

  • analysis of maximal continuous session length (humans have to rest) and correlation with time of day (see Distinguishing Humans from Bots in Web Search Logs)

It must be considered that bot characteristics exhibit a wide variability for different crawler / different sites, therefore it's difficult to derive simple, deterministic heuristics: rule based systems imply a long list of static rules that are difficult to define and maintain (even by experts).

Machine learning techniques are often used:

Almost every available AI/ML "tool" has been experimented. The main problem using these supervised machine learning tools is labeling the training dataset.

Even restricting the analysis to network-oriented behavior, this is a question of tremendous scope, for this reason I'm giving some keywords for further searches.

Notes

  1. Machine Learning based attacks are improving and CAPTCHAs also serves as a benchmark task for artificial intelligence technologies (e.g. The End is Nigh: Generic Solving of Text-based CAPTCHAs)

Keywords

HIP (Human Interactive Proofs), CAPTCHA, Keystroke dynamics, Keystroke cadence, typing dynamics, IDS (Intrusion Detection System), honeypot, click fraud, spambot

References

manlio
  • 2,092
  • 20
  • 30