Data Science

Airflow

Apache Airflow is a platform to programmatically author, schedule, and monitor workflows. 

Amazon EMR

Amazon Elastic MapReduce. A tool for big data processing and analysis. Provides a managed Hadoop and Apache Spark framework that makes it easier to process vast amounts of data across dynamically scalable Amazon EC2 instances.

Ambari

It is a software project for easier Hadoop management. It enables system administrators to provide, manage and monitor Hadoop clusters. It also provides features for installation and configuration of Hadoop services and a dashboard for monitoring cluster status.

Analysis

Analysis is the process of breaking a complex topic or substance into smaller parts in order to gain a better understanding of it.

Apache Avro

A data serialization system that relies on schemas for reading data. Using a schema helps cut back on serialization size. Avro also provides data structures, remote procedure call, compact binary data format and integration with dynamic languages. 

Apache Beam

A unified model for defining both batch and streaming data-parallel processing pipelines. Provides a general approach to expressing embarrassingly parallel data processing pipelines and supports end users, SDK writers, and runner writers.

Apache Flink

A general-purpose data processing platform and a top-level Apache project. It provides efficient, fast, accurate, and fault tolerant handling of massive streams of events. Flink is usable for dozens of big data scenarios and capable of running in standalone modeIts defining feature is its ability to process streaming data in real time.

Apache Hive

A data warehouse infrastructure built on top of Hadoop. It provides tools to enable easy data ETL, a mechanism to put structures on the data, and the capability for querying and analysis of large datasets stored in Hadoop files.

Apache Mahout

A machine learning framework for creating scalable applications, that can be used by data scientists, mathematicians and statisticians to implement their algorithms. Mahout also offers core algorithms that can be used for classification, clustering and batch based filtering.

Apache Pig

A programming framework used to analyze and transform large data sets. Apache Pig provides a high-level language known as Pig Latin which helps Hadoop developers write data analysis programs. By using various operators provided by the Pig Latin language, programmers can develop their own functions for reading, writing, and processing data.

Apache Spark

An open-source lightning-fast cluster computing technology, designed for fast computation. Has in-memory cluster computing that increases the processing speed of an app.

Apache Flume

A distributed, reliable and high availability service for collecting, accumulating and moving to a centralized repository of large amounts of streaming data from multiple sources. 

Apache Zeppelin

A web-based notebook tool for interactive data analytics. Zeppelin supports different technologies that aid in analytics, such as SQL, Python and Apache Spark. Aside from analytics, it can also perform discovery, ingestion, collaboration and visualisation.

Applied mathematics

Applied mathematics is the application of mathematical methods by different fields such as science, engineering, business, computer science, and industry. Thus, applied mathematics is a combination of mathematical science and specialized knowledge.

Artificial Intelligence

A field of study which determines ways of engineering a computer, a computer-controlled robot, or a software think intelligently, similarly to how humans are capable to think. It includes includes many disciplines, including those that study how human brain works, our learning, decision capabilities and behavior while trying to solve a problem.

AWS lambda

An event-driven, serverless computing platform provided by Amazon. Runs code in response to events and automatically manages the computing resources required by that code. Allows developers to program functions on a pay-per-use basis without having to provision storage or compute resources to support them.

Artificial Neural Networks

In machine learning and data mining, ANNs are computer systems that resemble the neural networks that constitute a biological brain. They are designed to learn tasks based on examples. For example, they can learn to identify certain types of images based on analyzing pre-labeled examples that define images as "this is a ..." and "this is not a ...".

Bayesian statistics

Bayesian statistics is a theory in the field of statistics based on the Bayesian interpretation of probability where probability expresses a degree of belief in an event.

Big Data

A term that describes the large volume of data – both structured and unstructured – that inundates a business on a day-to-day basis. Can be analyzed for insights that lead to better decisions and strategic business moves.

Big Data tools

Hadoop, Hive, Pig, Apache HBase, Cassandra, MapReduce (method), Spark.

Caffe

A deep learning framework, best used in image classification and segmentation, where speed, modularity and expression are important. Caffe can be implemented in different scale projects, from academic to industrial, as it can process more than 60M images in a day.

Computer vision

Computer vision is a field of computer science that works on enabling computers to see, identify and process images in the same way that human vision does, and then provide appropriate output.

Convolutional neural network

Convolutional neural network (CNN, or ConvNet) is a class of deep neural networks, most commonly applied to analyzing visual imagery. They have applications in image and video recognition, recommender systems, image classification, medical image analysis, and natural language processing.

Dataset

A collection of data that is organized into some type of data structure.

Data Modeling

Data modeling is the process of documenting a complex software system design as an easily understood diagram, using text and symbols to represent the way data needs to flow. 

Development by Synergize.digital

Sign up for updates
straight to your inbox