streamlit for a quick tool
so i needed a build a quick solution for content authors at work to clean up and transform a bunch of HTML files. take a bunch of html files → configure some things + add some some mapping from csv files → select which set of transformations → process and output a zip of files
i chose streamlit as it seemed like the most convenient option.
minimal syntax for the uploader and the config, beautifulsoup for processing html, it went smoothly.
as i was going through the documentation, i realised that streamlit is far more versalite than i thought and it’s not just a file uploader and buttons you can have on top of some python logic.
one particular section caught my eye and it’s about custom components
custom components (streamlit backend + react frontend)
so you can render custom frontends via react (or any frontend) in an iframe and bidirectional communication is possible.
this made me think of what sort of things i can build with this setup.
so in my work machine, i had this simple script in a jupyter notebook that traverses the codebase and copies all the files into a markdown file (respecting the .gitignore). i was using this to copy the context of the codebase and using with the internal LLMs for analysis and planning. this was basically a poor man’s version of repoprompt
knock-off repoprompt
so this streamlit + react setup felt correct to me for this use case for a bunch of reasons
- needs filesystem access (so it can’t just be a web app)
- well i was considering just a simple frontend that can take a zip file of the codebase and return the context
- but this means extra steps for users so i didn’t go with this.
- python GUI libraries (tkinter) felt clunky and i didn’t think i could come up with an intuitive interface for file selection
- electron is an option but it’s bulky and the means of distribution can get complicated inside the firm
- also pip install xyz can get the tool to the users system which felt like it’s easy installation through internal pypi artifactory.
- sleek and intuitive react frontend can have better ux
so i went ahead and created this tool which took me a few hours over a couple of days
- i was taking the path of the repo as an input and then i had file processing functions which also respected the .gitignore. also some heuristics to weed out unnecessary files (font files, lock files etc,.)
- then i processed and consolidated all of this info and sent it to the frontend. i added token counts for each files and the overall count.
- in the ui, i used shadcn library to have the file-tree structure with checkboxes to intuitively select the required files.
- i gave the option to preview the files and also the copy to clipboard function with markdown or text
this tool ended up being useful to me on multiple occasions and it felt nice to create this quickly with the help of llms.
shifting my workflow
this process changed my mind on how i think about prototyping solutions.
previously, i defaulted to react or next.js for any frontend things.
now i want to see if i can quickly prototype with streamlit or jupyter first no matter what sort of app it is. streamlit is excellent for simple and complex use cases as well. LLMs are also good at spitting out decent streamlit code.
once the rapid iteration and validation is done, i can always migrate it to a proper setup if needed.
how i would create the open source repoprompt now
so this streamlit + react tool happened a few months ago and after that, i was doing some research on how i can build an open source version of repoprompt.
without the limitations of a big firm internal process, this opens up more options.
based on my research, i will probably try to build it via one of these options
- VS code extension. you can select the relevant files right from the IDE. Also installation is straightforward.
- Tauri based app. Rust + React frontend (seems like a better option than electron)
- Native swift (not so familiar with this language, so could be a fun side quest)
conclusion
so yeah, streamlit is actually awesome. and jupyter notebooks are also awesome.
maybe consider these options for rapid prototyping before going for full-blown setups.