Data mining is the process of discovering patterns in large data sets involving methods at the intersection of machine learning, statistics, and database systems. Data mining is an interdisciplinary subfield of computer science and statistics with an overall goal to extract information from a data set and transform the information into a comprehensible structure for further use. Data mining is the analysis step of the "knowledge discovery in databases" process or KDD. Aside from the raw analysis step, it also involves database and data management aspects, data pre-processing, model and inference considerations, interestingness metrics, complexity considerations, post-processing of discovered structures, visualization, and online updating.
Business intelligence (BI) comprises the strategies and technologies used by enterprises for the data analysis of business information. BI technologies provide historical, current and predictive views of business operations. Common functions of business intelligence technologies include reporting, online analytical processing, analytics, data mining, process mining, complex event processing, business performance management, benchmarking, text mining, predictive analytics and prescriptive analytics. BI technologies can handle large amounts of structured and sometimes unstructured data to help identify, develop and otherwise create new strategic business opportunities. They aim to allow for the easy interpretation of these big data. Identifying new opportunities and implementing an effective strategy based on insights can provide businesses with a competitive market advantage and long-term stability.
According to Hotho et al. (2005) we can differ three different perspectives of text mining, namely text mining as information extraction, text mining as text data mining, and text mining as KDD process. Text mining is "the discovery by computer of new, previously unknown information, by automatically extracting information from different written resources." Written resources can be websites, books, emails, reviews, articles. Text mining, also referred to as text data mining, roughly equivalent to text analytics, is the process of deriving high-quality information from text. High-quality information is typically derived through the devising of patterns and trends through means such as statistical pattern learning. Text mining usually involves the process of structuring the input text, deriving patterns within the structured data, and finally evaluation and interpretation of the output. 'High quality' in text mining usually refers to some combination of relevance, novelty, and interest. Typical text mining tasks include text categorization, text clustering, concept/entity extraction, production of granular taxonomies, sentiment analysis, document summarization, and entity relation modeling.
Business performance management is a set of performance management and analytic processes that enables the management of an organization's performance to achieve one or more pre-selected goals. Gartner has officially retired the concept of "CPM" and reclassified it as "financial planning and analysis (FP&A)," and "financial close" to reflect two concepts: increased focus on planning, and the emergence of a new category of solutions supporting the management of the financial close.
Analytics is the discovery, interpretation, and communication of meaningful patterns in data. It also entails applying data patterns towards effective decision making. In other words, analytics can be understood as the connective tissue between data and effective decision making within an organization. Especially valuable in areas rich with recorded information, analytics relies on the simultaneous application of statistics, computer programming and operations research to quantify performance.
Semantic Research, Inc. is a privately held software company headquartered in San Diego, California with flagship offices in Washington, D.C. and Tampa, Florida. Semantic Research, is a California C-corporation that offers patented, graph-based knowledge discovery, analysis and visualization software technology. Its most popular product is a link analysis software application called SEMANTICA Pro.
Unstructured data is information that either does not have a pre-defined data model or is not organized in a pre-defined manner. Unstructured information is typically text-heavy, but may contain data such as dates, numbers, and facts as well. This results in irregularities and ambiguities that make it difficult to understand using traditional programs as compared to data stored in fielded form in databases or annotated in documents.
RapidMiner is a data science software platform developed by the company of the same name that provides an integrated environment for data preparation, machine learning, deep learning, text mining, and predictive analytics. It is used for business and commercial applications as well as for research, education, training, rapid prototyping, and application development and supports all steps of the machine learning process including data preparation, results visualization, model validation and optimization. RapidMiner is developed on an open core model. The RapidMiner Studio Free Edition, which is limited to 1 logical processor and 10,000 data rows is available under the AGPL license, by depending on various non-opensource components. Commercial pricing starts at $5,000 and is available from the developer.
Magic Quadrant (MQ) is a series of market research reports published by IT consulting firm Gartner that rely on proprietary qualitative data analysis methods to demonstrate market trends, such as direction, maturity and participants. Their analyses are conducted for several specific technology industries and are updated every 1–2 years.
Panopticon Software was a multi-national data visualization software company specializing in monitoring and analysis of real-time data. The firm was headquartered in Stockholm, Sweden. It partnered with several large systems integrators and infrastructure software companies, including SAP, Thomson Reuters, Kx Systems, and One Market Data (OneTick). The company's name is derived from the Greek: 'pan' for all, 'optic' for sight. The company name is derived from the word panopticon which is an architectural concept originally intended to facilitate surveillance of prisons.
Saffron Technology, Inc., a technology company headquartered in Cary, North Carolina, develops cognitive computing systems that use incremental learning to understand and unify by entity the connections between an entity and other “things” in data, along with the context of their connections and their raw frequency counts. This approach provides a semantic and statistical representation of knowledge. Saffron learns from all sources of data including structured and unstructured data to support knowledge-based decision making. Its patented technology captures the connections between data points at the entity level and stores these connections in an associative memory. Similarity algorithms and predictive analytics are then combined with the associative index to identify patterns in the data. Saffron’s Natural Intelligence platform is utilized by Global 1000 companies across industries including manufacturing, energy, defense and healthcare, to help decision-makers manage risks, identify opportunities and anticipate future outcomes, thus reducing cost and increasing productivity. Competitors include IBM Watson and Grok.
Platfora, Inc. is a big data analytics company based in San Mateo, California. The firm’s software works with the open-source software framework Apache Hadoop to assist with data analysis, data visualization, and sharing.
Feedzai is a data science company that detects fraud in omnichannel commerce. The company uses real-time, machine learning to analyze big data to identify fraudulent payment transactions and minimize risk in the financial industry.
Paxata is a privately owned software company headquartered in Redwood City, California. It develops self-service data preparation software that gets data ready for data analytics software. Paxata's software is intended for business analysts, as opposed to technical staff. It is used to combine data from different sources, then check it for data quality issues, such as duplicates and outliers. Algorithms and machine learning automate certain aspects of data preparation and users work with the software through a user-interface similar to Excel spreadsheets.
In the fields of information technology (IT) and systems management, IT operations analytics (ITOA) is an approach or method to retrieve, analyze, and report data for IT operations. ITOA may apply big data analytics to large datasets to produce business insights. In 2014, Gartner predicted its use might increase revenue or reduce costs. By 2017, it predicted that 15% of enterprises will use IT operations analytics technologies.
Luminoso, a Cambridge, MA-based text analytics and artificial intelligence company, spun out of the MIT Media Lab and its crowd-sourced Open Mind Common Sense (OMCS) project.
Embedded analytics is the technology designed to make data analysis and business intelligence more accessible by all kinds of application or user.
Amenity Analytics is an American software company founded in 2015 in New York City, United States. Amenity Analytics' cloud-based text analytics tools allow companies in any industry to systematically extract actionable intelligence from any text in real time. Amenity Analytics offers software products for text mining, sentiment analysis, and text analytics to automatically process structured data and unstructured data, using natural language processing, machine learning, artificial intelligence, and other technologies.
Guided analytics is a sub-field at the interface of visual analytics and predictive analytics focused on the development of interactive visual interfaces for business intelligence applications. Such interactive applications serve the analyst to take important decisions by easily extracting information from the data.
