Inspiration
Are autism and neuro-divergence mental deficiencies? Clinical therapy often treats it like so, but we believe otherwise. The challenges with clinical Applied Behavioural Analysis (ABA) for autism in the status quo include:
- Lack of meaningful social engagement with peers.
- Authority, taskmaster-based model for compliance.
- Specialists are uncommon and inaccessible.
- Behavioral improvements are focused on conformity, not individuality.
LEGO-based behavioral therapy is a clinically-studied and gamified approach to social competence therapy. It assigns social roles and encourages interactive gameplay during sessions, increasing intrinsic motivation, fostering authentic interaction with peers, and reducing stress/anxiety surrounding therapy. Teaching communication and self-regulation doesn't have to affect the intrinsic, unique qualities of our kids.
We wanted to find a way to improve and scale this idea, maximizing the creativity and independence of our next generation.
What it does
The generator intakes images (e.g., children's drawings) and outputs a 3D, layer-by-layer LEGO build with instructions suitable for behavioural exercises. Each instruction set contains separate requirements for a guide, supplier, and builder role in the LEGO construction.
How we built it
Front-end: React.js, Tailwind CSS, NeatJS, Three.js
Back-end: Python (Flask API, Gemini API, Shap-E, Modal)
Challenges we ran into
- Determining a sort algorithm to allocate LEGO blocks for our build (e.g., greedy algo, different bounding mechanisms).
- Colorizing our LEGO designs after voxelization.
- Connecting our pipeline: frontend interfaces, backend APIs, and AI model workflows.
Accomplishments that we're proud of
- Developing a brick-placement algorithm leveraging border detection and structural integrity principles.
- Integrating Gemini into our workflow, enabling dynamic AI-generated content.
- Trouble-shooting errors in the "pipeline," being resilient de-buggers.
- End-to-end system integration, managing API redirects and making generated outputs (files, text from Gemini, etc.) accessible within the application.
- Winning the Best AI for Creativity & Generative Arts award at the GenAI Genesis 2025 Hackathon, the largest Canadian AI hackathon.
What we learned
- Conversion algorithms: color transfer to voxelization and mesh to voxel algorithms.
- Generative 3D model architectures (Shap-E): including latent diffusion models and implicit neural representations.
- How to work with new 3D file formats, including .obj, .ldr, and .stl.
What's next for LegoFIKS
- Expanding build pieces (incl. LEGO Library, custom pieces), fine-tuning structures.
- Generating instructions for single/multi-builder modes with behavioural principles.
- Integrating further clinician insights: creating simultaneous platforms for school/home users and licensed healthcare professionals to work in conjunction.
Demo of LegoFIKS in action.