Sun, Jan 5, 2025. Every few years, something comes along trying to replace relational databases and SQL, and gets absorbed. YouTube#database
Key value stores. People soon realize they need more features, e.g. indices.
MapReduce systems. Most MapReduce vendors put SQL on top of SQL. Then the Hadoop market crashed. (But HDFS, S3, distributed storage systems are a good idea)
Document Databases. JSON. SQL absorbed that. SQLite 3.45+ supports even JSONB. DuckDB, of course, has JSON.
Array Databases. SQL:2023 adds SQL/MDA which allows for matrix operations. But specialized databases might make sense in this category.
Vector Databases. Every DB is adding support for this.
Tue, Nov 26, 2024. LLMs may be good at derived data generation. For example, given a database schema, what derived columns would be useful? What derived views would be useful? #database#llm-ops
Sat, Feb 17, 2024. Discussion with Vinu Yamunan #dev#database
Databuck by FirstEigen. Autolysis plus monitoring
Quality council has the data steward (maintainer of each dataset) coming together with the uses on a weekly basis to understand what quality problems to users are facing. Data owners jaundice at a lower frequency to get an understanding
#TODO Automate rules for data quality in our projects and intranet
Convert a config rule into business language. Explain SQL. These are good use cases for llm's
Graph DBs are powerful for flexible data structures, but query generation needs AI or expertise. Check the Neo4J language cypher
Explore storing SAME data in relational DBs AND in graph DBs / document DBs for different use cases
Dallas rocketry challenge. Build a rocket that can take an egg to 800 feet exactly and land without breaking it
Tue, Jan 16, 2024. Lilac is an LLM-based data curation tool. Use it to search by concept (e.g. PII, duplicates, etc.) and then drop/update the results. #ai-coding-tools#database#llm-ops
Thu, Dec 21, 2023. Google Knowledge Graph ID powers the Knowledge Graph. If it begins with /m/ it's the same as the FreeBase ID. This is now available as WikiData. e.g https://www.wikidata.org/wiki/Property:P2671#database