3. Analysis
We consider both qualitative and quantitative parameters as illustrated in Diagram 1 and summarized below in 4 separate Tables 1- 4.
It’s important to note that Tables 1 - 4 are generic: many higher-order details are intentionally not included so that the main structure remains visible. Also, creativity of AI contribution to the final result depends on the level of creativity required.
1. Most qualitative output parameters (Table 1) require human or expert judgment, although some can be supported by rubrics, comparative baselines, user-response data, linguistic analysis, or AI-assisted scoring.
|
Because creativity is first a question of value and meaning, qualitative parameters are presented before quantitative parameters. Quantitative measures are useful, but they become informative only when interpreted through qualitative criteria such as originality, usefulness, depth, coherence, resonance, and generativity.
Table 1. Qualitative Parameters Characterizing Creativity of the Result/Output
|
|
No.
|
Core Qualitative Parameter
|
What It Asks
|
Assessment Approach
|
Why It Matters
|
|
1
|
Originality and surprise
|
Is the result fresh, non-obvious, and productively unexpected?
|
Human or expert rating; comparison with baseline/common responses; semantic-distance support.
|
Captures the basic novelty signal of creativity.
|
|
2
|
Usefulness and fit
|
Does the result address the task, audience, context, and larger purpose?
|
Task-fit rubric; expert/domain review; user acceptance or adoption.
|
Prevents creativity from meaning merely strange or fluent.
|
|
3
|
Conceptual depth and insight
|
Does the result reveal hidden structure, reframe the problem, or help the reader see differently?
|
Expert rating; depth/reframing rubric; explanatory-value assessment.
|
Separates surface variation from meaningful creative thinking.
|
|
4
|
Coherence and elegance
|
Do the parts fit together clearly, simply, and with minimal unnecessary complexity?
|
Coherence rubric; parsimony review; expert judgment.
|
Strong creative outputs are not only novel; they also hold together.
|
|
5
|
Expressive power and voice
|
Is the idea communicated vividly, precisely, memorably, and in a fitting voice?
|
Writing-quality rubric; style comparison; reader/audience response.
|
Packaging shapes whether the idea is understood, remembered, and continued.
|
|
6
|
Generativity and transformative potential
|
Does the result open new directions, stimulate follow-up ideas, or change the path of further work?
|
Follow-up idea tracking; user/expert forecast; downstream-use or adoption evidence.
|
Measures whether the output becomes a source of future creativity.
|
|
7
|
Resonance and continuation
|
Does the result attract attention, invite continuation, and make collaborators want to keep working?
|
User/reader rating; continuation behavior; follow-up prompt or idea tracking.
|
Captures whether the output energizes the next round of creative work.
|
2. Analyzing Quantitative Parameters Characterizing Creativity of the Result/Output (Table 2), it’s important to keep in mind that more is not always better. A high number of generated ideas may reflect fluency, but not necessarily originality or usefulness. Similarly, longer responses may seem more engaged while adding redundancy. Quantitative measures therefore need to be paired with qualitative interpretation.
Table 2. Quantitative Parameters Characterizing Creativity of the Result/Output
|
No.
|
Quantitative Parameter / Measure
|
Interpretive Category
|
Assessment Approach
|
Why It Matters
|
|
1
|
Number of distinct ideas
|
Fluency
|
Human coding; automated idea segmentation with human check.
|
Captures productive breadth, while requiring quality interpretation.
|
|
2
|
Number of idea categories or conceptual clusters
|
Flexibility / conceptual diversity
|
Human coding; clustering; topic-modeling support.
|
Measures range and movement across idea space rather than sheer volume.
|
|
3
|
Normalized relevant conceptual distance
|
Novelty with relevance control
|
Embedding-based distance from prompt, baseline, or common response pattern, normalized to task/domain context and filtered by human relevance/usefulness rating.
|
Prevents distant-but-useless ideas from being overvalued as creative.
|
|
4
|
Useful elaboration units per idea
|
Development / elaboration density
|
Count of relevant examples, subpoints, mechanisms, or supporting relations per distinct idea, with human review.
|
Distinguishes developed ideas from long but thin output.
|
|
5
|
Number of meaningful structural units or links
|
Organized complexity
|
Count sections, levels, argument links, dependencies, or cross-references; validate by human coding.
|
Captures structure and integration, not just length.
|
|
6
|
Number of coined terms or new constructs per section/page
|
Conceptual invention
|
Human coding; keyword extraction; glossary tracking.
|
Tracks creation of reusable conceptual tools.
|
|
7
|
Number of follow-up ideas, questions, revisions, or prompts triggered
|
Generativity / uptake
|
Reaction logs; reader or AI follow-up coding; downstream-use tracking.
|
Measures whether the output becomes a seed for further creative work.
|
3. The output-based indicators in Tables 1 and 2 should be supplemented, where possible, by computational-behavior indicators (Tables 3 and 4). These would not be available to ordinary users in most commercial systems, but they are important for experimental research. Possible indicators include number of active nodes or components involved in producing a response, distribution of activation across layers, changes in attention patterns, variation in weight use or routing behavior, correlation and auto-correlation patterns across internal representations, and time-dependent signatures of generation. In this sense, AI e-xcitement can be treated as a metaphor for measurable changes in internal computational dynamics rather than as a claim about subjective feeling.
Table 3. Qualitative Parameters Characterizing the Creative Process
|
No.
|
Core Qualitative Process Parameter
|
What It Asks
|
Assessment Approach
|
Why It Matters
|
|
1
|
Creative attention and investment
|
Does the process show sustained allocation of attention, effort, and creative resources to the task?
|
Session review; density of substantive comments; persistence through difficulty, boredom, uncertainty, or digression.
|
Creativity requires invested attention; playful detours or “quantum jumps” may support rather than reduce creative work.
|
|
2
|
Trajectory quality
|
Do idea shifts, detours, and returns form a meaningful path relative to the task?
|
Trajectory mapping across major idea states, decisions, rejected options, and revised versions.
|
Distinguishes productive exploration from random drift without requiring a straight-line path.
|
|
3
|
Selection and judgment quality
|
Are ideas kept, changed, rejected, combined, or delayed for clear creative reasons?
|
Process log; transcript coding; comparison of candidate ideas with retained versions.
|
Creative process depends not only on idea generation but on meaningful selection under constraints.
|
|
4
|
Revision depth
|
Does the process improve the work at the level of meaning, structure, framing, or conceptual clarity?
|
Draft comparison; coding of surface edits versus substantive transformations.
|
Separates real creative development from cosmetic polishing.
|
|
5
|
Reframing and emergence
|
Does the process generate new frames, concepts, or possibilities that were not available at the start?
|
Turning-point analysis; idea genealogy; annotation of new concepts, metaphors, or problem definitions.
|
Captures the transformative side of creative process.
|
Table 4. Quantitative Parameters Characterizing the Creative Process
|
No.
|
Quantitative Process Parameter / Measure
|
Access Level
|
Assessment Approach
|
Why It Matters
|
|
A. User-accessible / external process traces
|
|
1
|
Number of substantive turns or work cycles
|
User / transcript
|
Count substantive user and AI turns, excluding purely administrative turns when appropriate.
|
Provides a basic trace of interaction intensity without equating length with quality.
|
|
2
|
Number of revision cycles or draft versions
|
User / document history
|
Version history, draft comparison, or session log.
|
Captures iteration and return to the artifact.
|
|
3
|
Number of constraints, criteria, or examples per prompt
|
User / prompt text
|
Prompt coding; count explicit requirements, exclusions, examples, and quality criteria.
|
Measures steering information supplied to the creative process.
|
|
4
|
Number of accepted, rejected, revised, or deferred ideas
|
User / transcript / drafts
|
Decision log; transcript coding; comparison of candidate ideas with retained text.
|
Quantifies selection and judgment activity.
|
|
5
|
Number and scale of substantive changes across versions
|
User / document comparison
|
Classify edits as surface, structural, conceptual, framing-level, deletion, or merger.
|
Distinguishes cosmetic editing from meaningful transformation.
|
|
6
|
Number of reframing events or new concept introductions
|
User / analyst coding
|
Turning-point annotation; idea genealogy; glossary or concept tracking.
|
Tracks observable emergence of new frames, terms, or directions.
|
|
7
|
Response latency or processing time, where available
|
System-visible / logs
|
Timing records or system logs, interpreted cautiously.
|
May reflect task complexity or system load, but is not a direct creativity score.
|
|
B. Developer/researcher-accessible / internal computational traces
|
|
8
|
Activation density and distribution
|
Instrumented research access
|
Measure number, proportion, and layer distribution of active units/components during generation.
|
Possible internal correlate of computational involvement or e-xcitement.
|
|
9
|
Attention, circuit, or module recruitment
|
Instrumented research access
|
Track number, strength, specialization, or routing of attention heads, circuits, modules, or pathways.
|
Indicates which internal resources participate in the task.
|
|
10
|
Memory/context resource allocation
|
Instrumented research access
|
Measure context use, retrieval activity, cross-turn carryover, or working-memory-like persistence.
|
Connects sustained creative process to memory and context management.
|
|
11
|
Long-range correlation or autocorrelation patterns
|
Instrumented research access
|
Analyze correlations across internal states over token time, layer depth, or generation windows.
|
Provides a possible “AI brain-wave-like” operational signature.
|
|
12
|
Persistence or stability of internal patterns
|
Instrumented research access
|
Track whether activation or correlation patterns persist across prompts, revisions, or task phases.
|
May indicate continuity of computational engagement across the creative session.
|
|
13
|
Routing or search breadth
|
Instrumented research access
|
Measure candidate pathways, branches, retrievals, tool calls, or internal alternatives where accessible.
|
May capture breadth of internal exploration during creative generation.
|
|
