It goes without saying that a fantastic web company requires a super data processing architecture that can provide a model or vision for the interaction of the company’s data systems. Data architecture guides the processing, storage, and utilization of data in an information system and determines the conditions for data processing operations, which enables the designing of data flows and optimizes control over the flow of data within the system.
Let’s see what architectures power some of the top web companies of the world:
Offline Data Composition: Nextdoor is a free social networking website that helps neighbors to get to know each other by talking online and build safe and supportive areas to live in.
Nextdoor is powered by offline data composition. The structure does not demand for a second-by-second update, a couple of hours old data works fine. More than 1000 agencies in the U.S. have partnered with it to accurately geo-target content to maintain the relevancy and accuracy of data. Among the various tools used to ensure this, there is one by the name of ‘Maps and Metrics page’. After the SLA (Service Level Agreement) is defined, the page lets agencies target content by showing anonymous coverage and engagement of an area represented on a map.
Data Analytics Platform: Netflix provides streaming movies and videos across the globe. The viewing service is divided into slateless and slateful tiers. The slateful tier stores the latest data for active views. All persistent data is stored in Cassandra. Memcached, layered on top of Cassandra, guarantees a low latency read path for materialized, perhaps stale, views of the data.
Netflix began with a slateful architecture design that preferred consistency over availability, in an environment of network partitions. It also pioneered running memcached and Cassandra in the cloud, where a slateful architecture helped them mitigate the risk of failure. However, there was a huge downside: the failure of even a single slateful node would deprive 1/nth of the member base of reading from or writing to their viewing history.
Analytics & Reporting Infrastructure: 500px is a Canadian online photography community and marketplace. It uses the log search engine Splunk to record the number of likes a photo has got in a specific time period. Splunk lets you search and analyze timestamped logs so that user behavior regarding a particular entity can be gauged.
MySQL was another option that was offered to 500px. However, it was declined since it isn’t effective in this scenario since it can only give the number of likes on a photo, but not information like how many likes it got in the last hour. Hence, it does not measure up to the requirement in analyzing user behavior.
Why is knowing about the data architectures of successful companies important?
Having information about how the best in the world plan their architecture provides immense support in building your own architectures by enabling you to better understand your needs and anticipate future requirements (or problems).
If you want to know more about sas cources, hadoop training, big data training and other analytics training visit Analytixlabs.