Wow, super interesting deep dive into the architecture design considerations, iterations, and final results. The visit to the data lab in Seattle sounds like a hackathon with all the whiteboarding and testing :O Also, I'm here because a Nasdaq exec talked about Redshift at a panel at Grace Hopper, which is happening right now! Cool stuff!!
@franckleveneur8134
3 жыл бұрын
I'm not surprised that Nasdaq ran into issues with Redshift. One of the issue I see is their Architecture seems to be poorly designed. Looks like they're loading OLTP tables and doing joins against it. That's basically moving OLTP tables to a System that is meant for OLAP. I'm curious if they are using dimensional modeling? Fact and Dimensions? Your temp table might save you for a while but you'll hit a snap soon. The issue I see is incorrect architecture for their data warehouse. They should probably take a look at DBT too. And i don't understand why you would go query a billion row table. Most of Nasdaq tables can probably be summarized (aggregated) hourly, daily, or weekly.
Пікірлер: 2