FOR INDUSTRIAL DESIGNERS
AN AI SKETCHING TOOL
SMART SKETCH
AI is becoming common in design, making sketching faster but often adding unwanted details. Designers want to keep creative control while enjoying the efficiency of AI. This project introduces an AI-assisted sketching tool that blends hand sketching with AI for work efficiency while retaining creative control.
This is an individual thesis project, where I served as the sole designer responsible for the entire design process, with guidance from an advisor.
TIMELINE
12 Months
Spring 2024 - Winter 2024
TEAM
UI/UX Designer
UX Researcher
Project Manager
MY ROLE
UI/UX Designer​
UX Researcher
Project Manager
TOOLS
Figma
FigJam
CONTEXT
Generative AI tools are often promoted as powerful aids for creative work, yet it’s important to examine the shortcomings that can occur when relying on them for ideation. Here are some examples, sourced from the official website of Prome AI, which were presented as successful case studies. While these examples showcase impressive abilities in rendering high-quality images from rough sketches, they also reveal critical mistakes in understanding and interpretation that cannot be overlooked.
Example 1
Adding extra details to the outsole, and misinterpreting the holes on the side?
PROBLEM
AI is increasingly used in design, speeding up sketching and ideation but often adding unnecessary details and skipping essential thought processes for designers. This project offers an AI-assisted sketching tool that combines the efficiency and idea generation of AI with the creative control and thoughtfulness of traditional hand sketching.
Work flow of hand sketches
Work flow of using generative AI
Proposed work flow
PROBLEM STATEMENT
Current generative AI interactions tend to diminish the creative role of designers during the ideation process. How can we design a workflow that fosters collaboration between human designers and AI, retaining AI’s benefits while preserving creative expression?
DESIGN SOLUTION​
PERSONALIZATION
During the sign-up process, users are prompted to answer a few experience-based questions and upload samples of their previous work. This information is used to train the AI model and deliver personalized, customizable assistance.
AI ASSISTANT
By collaborating with the AI assistant, users can utilize the chatbot for a variety of tasks, such as creating color swatches, generating reference images, activating specific tools... By enabling these complex tasks to be completed through simple, intuitive conversations, this feature facilitates seamless and dynamic collaboration with the AI in a highly user-friendly manner.
PERSPECTIVE GRID
The perspective grid, one of the AI tools, automatically generates guidelines to assist users during sketching. AI analyzes the initial lines drawn by the user to generate an accurate grid for reference. Once the sketch is completed on the grid, users can drag and rotate the canvas to explore a different view, dynamically adjusted by the AI for an alternative perspective.
FOCUS ADVICE
The focus advice tool lets users crop a canvas area for localized edits, with options to adjust style and provide text input for customization. This feature mimics the collaborative dynamic of a student and instructor during sketching. By cropping a specific area, users can spotlight a region for advice, and the AI generates multiple options for users to explore and choose from.
ACTIVE FEEDBACK
Active feedback lets the AI critique work, mimicking team comments on a shared document. Users have the option to activate this feature continuously to receive live AI feedback as they sketch or turn it on periodically to review new comments. This flexibility empowers users to decide how much they want the AI to participate in their creative process, tailoring the experience to their personal preferences.
SO, HOW DID WE GET THERE?
To visualize design need
Let's do a fun activity
EXPLANATION
The exercise above serves as a metaphor for the ideation process when designers sketch and create with Generative AI.
​
If the image appears convincing to you, it is actually an AI-generated result rather than a real photo, revealing key inaccuracies that do not align with reality.​
When compared to an actual photo of giraffes drinking water, as shown on the right, the AI-generated image exposes flaws in the leg positioning and the proportion of the head and neck to the body.
​
This exercise highlights that translating an idea into a visual form requires meticulous scrutiny of each detail. If an incorrect AI-generated result is presented before the idea is fully formed, it becomes easy to accept the AI’s version, even when it deviates from the original intent.
​
This project aims to develop an AI sketching tool that enhances collaboration between human designers and AI, optimizing workflow efficiency while ensuring creators maintain full control over the ideation process.
DESIGN PROCESS
DESKTOP RESEARCH
Technical difficulties in design sketching
Designers face multiple technical challenges during the sketching process, and the design goal is to incorporate AI-assisted tools that can help address these difficulties.
Key values of design sketching
These points represent the key value of the design sketching process, and Generative AI should not replace the designer's thinking process but rather reinforce it, providing support through these critical touchpoints.
USER INTERVIEW
To understand how designers navigate the challenges of sketching and realize the core values of ideation, as well as to assess the effectiveness of current AI tools in supporting this process, interviews are conducted to gather insights into their experiences, opinions, and needs.
Recruitment
Most interview participants were recruited through personal networks of the researchers.
All interviewees are designers or design students who are familiar with design sketching and generative AI.
Interview Details
There were 6 interviews conducted both virtually and in-person.​
Each interview took between 30-45 minutes.
Goal
-
To explore designers’ habits of using AI-generated images versus hand sketches to generate ideas in their design work flow.​​
-
To understand why designers adopt generative AI or not and how they perceive the differences.​
-
To gather insights of potential design opportunities.
Affinity Map
User responses were documented and key viewpoints were noted on Post-it notes. These notes were organized in a matrix: the x-axis represented the project's four design values—creative control, thinking process, work efficiency, and idea generation—while the y-axis compared hand sketching with generative AI. Notes were color-coded, with blue for positive feedback and red for negative. By counting these opinions, the analysis highlights designers' preferences for image generation based on their prioritized design values.
Journey Map
User’s goal at each step
Interaction with a human instructor
The map outlines the student's goals at each stage, such as learning tools, refining sketches, and grasping depth and composition. It also tracks their emotions, from early excitement and confidence to frustration and eventual satisfaction as challenges are tackled.
​
Documented interactions between the student and instructor show how guidance is given, covering setup, tool use, corrections, verbal instructions, and demonstrations. This detailed breakdown helps translate human support into AI-driven features, emphasizing step-by-step guidance, corrective feedback, and emotional support throughout the creative process.
COMPETITOR ANALYSIS
Previous attempts to integrate AI into the creative process often focus on automating the rendering process rather than enhancing the designer’s manual input during the crucial ideation phase. The proposed project differs by focusing on augmenting, not replacing, the traditional sketching process, making AI a co-creator rather than merely a tool for post-ideation rendering.
DALL-E
Relying solely on text descriptions for creation lacks the precision that creative professionals require.
PromeAI
Remaining as a tool for generating outputs, but not assisting in early ideations
Midjourney
Focusing on creating art with more stylistic and aesthetic elements, rather than design and functionality
Krita
Suiting to fine artists rather than designers who need precision and conceptual clarity
Stable Diffusion
Restricting creative control to only text inputs and preset parameters, no hands-on adjustment
Hypersketch
Limiting AI's creativity to rendering, offering little support in early design stages
Vizcom
Requiring a finished sketch to render, without leveraging AI as an active creative partner
Design Opportunity
Augmenting traditional sketching process, making AI a co-creator rather than for post-ideation rendering only
While many incorporate AI for enhancing visuals or aiding in rendering, they tend to focus on applying AI after the initial creative ideation is already done. They don’t provide a meaningful interaction where AI and designers work collaboratively in the foundational phase of ideation and sketching.
The competitive analysis matrix highlights the positioning of current AI-powered sketching tools in terms of creative control and the level of cognitive engagement required from the designer.
​
The blue ocean opportunity lies in the quadrant that combines high creative control with a high requirement for the designer’s thinking process. This suggests a potential gap in the market for tools that fully empower designers to maintain their creative intent as well as decision making opportunity while improving work efficiency through AI assistance.
Objectives
Problem statement
Current generative AI interactions tend to diminish the creative role of designers during the ideation process. How can we design a workflow that fosters collaboration between human designers and AI, retaining AI’s benefits while preserving creative expression?
Hypothesis
Breaking down creative process into smaller, well-defined tasks and integrating AI as collaborative partner will enhance creativity and efficiency by reducing cognitive load and supporting iterative collaboration.
Design objective
To design a UI/UX experience that integrates AI to assist users in defining, ideating, and critiquing throughout the creative process, promoting a more creative and effective workflow through structured, smaller steps.
IDEATION MATRIX
Concept 1
A tablet based sketching tool with multi AI touch points and dynamic user flow
Concept 2
A computer based AI image generator with high customization
Concept 3
A VR based sketching assistant that gives virtual AI tips when drawing on paper
Based on the established scoring system, the concept of a tablet-based sketching tool featuring multiple AI touch points and a dynamic user flow emerged as the highest-scoring idea, with an average rating of 4.5. This concept not only achieved the highest overall score but also led in the most critical criterion: its potential to foster creativity among users. Given its strong performance, especially in promoting creativity, a key objective of this project, this concept has been selected for further development.
Prototypes
Rough sketch
Lo-fi prototype
Mid-fi prototype
Prototype flow
MID-FI USER TESTS
User Test Abstract
Task Completion
REVISIONS
1. Chat Bot Logo
The first significant change involves the entrance icon for the AI chatbot. During user testing, two participants were unable to complete this task, citing difficulty in identifying the chatbot’s entry point. In the original design, the AI chat feature was integrated alongside other AI tools, making it challenging for users to distinguish it. To address this issue, the interface was updated by introducing a floating icon positioned prominently above the canvas, making the feature more noticeable. Additionally, a text label was added to further clarify its purpose.
2. Perspective Grid Adjuster
The second change focuses on improving the perspective grid adjustment feature. In the original design, users adjusted the perspective grid by manipulating parameters for the x and y positions of the vanishing points. However, user testing revealed that this interaction was not intuitive enough—one participant failed to complete the task, and several others provided feedback questioning the approach. Based on participant suggestions, the new interaction allows users to directly drag on the canvas to adjust the grid.
3. Focus Advice Input
The third improvement involves adding several parameters to the focus advice input feature. In the original design, the only inputs available were a text description and a checkbox for maintaining style consistency with the entire sketch. However, user testing participants expressed a desire for more controllable options within this feature. Inspired by tools such as Stable Diffusion and Vizcom, additional parameters were introduced to give users more control over the outcome. In the updated design, users can adjust various aspects such as style, shape, and rendering effect, allowing them to better define their desired results and achieve a more customized output.
LOGO DESIGN
The logo has multiple layers of meaning that reflect the brand’s core values. The “SS” symbol stands for “Smart Sketch” and serves as a recognizable identifier. The series of ascending steps represents the step-by-step AI assistance provided by the app, emphasizing that the AI is meant to guide users through their creative journey progressively, rather than generating results in a single step. This layered approach to the logo design reinforces the idea that Smart Sketch is here to support artists incrementally, making their creative process more manageable and intuitive.
VISUAL GUIDELINE
INFORMATION ARCHITECTURE
SIGNIFICANT FEATURES
HI-FI USER TESTS
To evaluate whether the changes made after Mid-Fi user tests are effective, hi-fidelity user tests were conducted in-person. In this round, 8 tests were conducted, each session lasted between 25 to 30 minutes, focusing primarily on assessing the effectiveness of the most current changes and the user flow over all. The tests utilized a clickable prototype built in Figma, presented to users on an iPad for a realistic experience.
User Test Abstract
User Test Photos
The user test included eight participants completing four tasks to assess the AI sketching app. All tasks were completed within 20 to 70 seconds. Compared to the previous test results, failure occurred only once for one participant, and the average time spent on each task decreased significantly. These results demonstrate that the modifications implemented after the last test have effectively enhanced the seamlessness of human-AI collaboration.
Hi-Fi Completion
Mid-Fi Completion
The System Usability Scale (SUS) is a trusted evaluation tool used in design to measure the usability of a product or system through user feedback. For this project, the SUS survey serves as an essential evaluation method, offering a quantitative measure of how well Smart Sketch facilitates human-AI collaboration.
​
During the survey, participants were asked questions about their collaborative experience with AI throughout the sketching process. Questions included statements like, “AI’s presence is taking over the whole creative workflow (reverse scoring)” and “I found the various functions in this system to be well integrated and helpful in easing my struggles while sketching.” By responding to these prompts, participants were able to share their perspectives on how effectively the product facilitates a human-AI collaborative creative workflow.
​
The collected survey results were analyzed, yielding an average score of 85, which exceeds the threshold for an A-grade evaluation. This indicates that most users found the interaction effective in fostering a balanced and productive human-AI collaboration within a professional creative environment for designers.
SUS Survey Result
