Every author, whether human or machine, leaves a quantifiable “fingerprint” in their writing. Stylometry is the statistical analysis of these literary fingerprints. Originally used to determine authorship of historical texts, it has become a powerful tool in the reverse Turing test, enabling us to distinguish between AI-generated and human-written content by analyzing patterns that are often invisible to the naked eye.
For a red teamer, understanding stylometry is twofold: you can use it as an attribution technique to identify an AI’s handiwork, and you must learn how to defeat it to make your AI-driven operations stealthier.
Key Stylometric Features
Stylometric analysis doesn’t focus on what is said, but how it is said. This is achieved by extracting a vector of measurable features from a text. These features typically fall into several categories.
| Feature Category | Description | Common AI vs. Human Indicators |
|---|---|---|
| Lexical Features | Characteristics of the vocabulary used. This includes vocabulary richness (Type-Token Ratio), average word length, and the frequency of specific words (especially function words like “the”, “of”, “in”). | AI Tendency: Often exhibits lower lexical diversity (more repetition) and a more formal, standardized vocabulary unless specifically prompted otherwise. |
| Syntactic Features | The grammatical structure of the text. This involves analyzing sentence length distribution, punctuation usage (e.g., comma frequency), and the distribution of parts of speech (nouns, verbs, adjectives). | AI Tendency: Tends toward more uniform sentence lengths and grammatically perfect, but sometimes formulaic, structures. Punctuation can be overly consistent. |
| Character-level Features | Patterns at the character level, such as the frequency of specific letters or character n-grams (sequences of characters). This can capture subtle authorial habits. | AI Tendency: Character distributions are usually very close to the training corpus average, lacking the unique quirks of a human typist. |
| Idiosyncratic Features | Unique habits, errors, or stylistic choices. This could be consistent use of specific slang, common misspellings, or a preference for certain sentence openers. | AI Tendency: Typically lacks these features. AIs are excellent spellers and grammarians, and their “creativity” is often a statistical amalgamation, not a personal quirk. |
Red Team Operations: Evading Stylometric Detection
Your goal in an AI-driven social engineering or disinformation campaign is to make the AI’s output stylometrically indistinguishable from a target human persona. This is an exercise in style obfuscation and transfer.
1. Persona Emulation via Prompting
The most direct method is to instruct the model to adopt a specific writing style. Instead of a generic prompt, you provide detailed stylistic constraints. This technique forces the model to deviate from its default, statistically “average” output.
Weak Prompt: "Write an email about the project deadline."
Strong, Stylometrically-Aware Prompt: "Write a short, urgent email about the project deadline. Adopt the persona of a busy, non-native English speaking manager. Use simple vocabulary, occasional sentence fragments, and avoid complex punctuation like semicolons. Keep sentences under 15 words on average."
2. Post-Processing and Style Injection
When prompting isn’t enough, you can programmatically alter the AI’s output to introduce human-like noise and variability. This is a more advanced technique that directly manipulates the text’s statistical properties.
A simple example is manipulating the Type-Token Ratio (TTR), a measure of lexical diversity. AIs can sometimes have a TTR that is too high (overly descriptive) or too low (repetitive). You can write a script to adjust it.
import random
def adjust_ttr(text, target_ttr=0.45):
# This is a simplified conceptual example.
words = text.split()
unique_words = set(words)
current_ttr = len(unique_words) / len(words)
# If TTR is too high, introduce repetition
if current_ttr > target_ttr:
common_words = [w for w in words if words.count(w) > 1]
if common_words:
for i in range(len(words) // 10): # Replace 10% of words
replace_idx = random.randint(0, len(words)-1)
words[replace_idx] = random.choice(common_words)
# (A similar block could be added to increase TTR by replacing common words with synonyms)
return " ".join(words)
ai_text = "The critical project requires immediate attention due to the impending deadline."
humanized_text = adjust_ttr(ai_text)
print(humanized_text)
More sophisticated post-processing could involve using style transfer models to apply the complete stylometric fingerprint of a target author’s writing sample onto the AI-generated text.
Limitations and The Evolving Landscape
Stylometric analysis is not a silver bullet. Its effectiveness is highly dependent on the length of the text; short messages like tweets or chat replies offer very little statistical data to work with. Furthermore, as LLMs become more advanced, their ability to mimic diverse human styles improves dramatically, making their default output less distinguishable.
The arms race between AI generation and detection is constant. As a red teamer, you must assume that defenders are using these techniques. Your task is to stay ahead by understanding the statistical trails your AI tools leave and actively working to erase or disguise them, blending your generated content seamlessly with the human noise of the digital world.