Application Log Analytics

A software company runs about two hundred microservices that together produce several terabytes of JSON application logs a day. Engineers want to search last month's logs with plain SQL when they investigate an incident, perhaps a few dozen queries a week, without keeping a database cluster running around the clock for such occasional use. Logs must be kept for one year at the lowest possible storage cost and should arrive from the services within a few minutes. The team does not want to write or run code that batches and uploads files, and queries should only pay for the data they actually scan.