Andrew Kim

Portfolio

Consulting engagements, projects, and open-source contributions.

ConsultingProjectsOpen source
2 contributions Clear filter

  • Dagster

    I resolved a long-standing issue where dynamically mapped Dagster steps failed to load ML models from S3, originally reported by a user who noticed Dagster's dynamic step names contain square brackets which S3 accepts, but Spark interprets as glob wildcards, making files unreadable despite being present in storage.

    My fix finally automated path sanitization for dynamic mapping, unblocking anyone persisting ML models or serialized objects to S3 and removing the need for users to build their own workarounds.

    Links: Pull request|Issue

    Dagster

  • LlamaIndex

    I added support for pathlib objects to one of the most commonly used data loading modules in LlamaIndex, the SimpleDirectoryReader, after I noticed it only accepted string paths.

    Links: Pull request|Issue

    LlamaIndex