Among the list of built-in (AKA first-party) algorithms are two topic modeling algorithms: Amazon SageMaker Neural Topic Model (NTM) and Amazon SageMaker Latent Dirichlet Allocation (LDA).
Amazon SageMaker Neural Topic Model (NTM)Īmazon SageMaker is an end-to-end machine learning platform that provides a Jupyter notebook hosting service, highly scalable machine learning training service, web-scale built-in algorithms, and model hosting service. For example, topic models can also be used for modeling other discrete-data use cases such as discovering peer-to-peer applications on the network of an internet service provider or corporate network. Although we focus on text documents here, the observations can be applied other types of data. It’s also worth mentioning that, topic modeling is a general algorithm that attempts to describe a set of observations with the underlying themes.
Topic modeling can also be used as a feature engineering step for downstream text-related machine learning tasks. There are many practical use cases for topic modeling, such as document classification based on the topics detected, automatic content tagging using tags mapped to a set of topics, document summarization using the topics found in the document, information retrieval using topics, and content recommendation based on topic similarities. The figure that follows shows the relationships among words, topics, and documents. For example, a collection of documents that contains frequent occurrences of words such as “bike,” “car,” “mile,” or “brake” are likely to share a topic on “transportation.” If another collection of documents shares words such as “SCSI,” “port,” “floppy,” or “serial” it is likely that they are discussing a topic on “computers.” The process of topic modeling is to infer hidden variables such as word distribution for all topics and topic mixture distribution for each document by observing the entire collection of documents. The technical definition of topic modeling is that each topic is a distribution of words and each document is a mixture of topics across a set of documents (also referred to as a corpus).
#An introduction to statistical learning amazon full
In addition, Amazon SageMaker NTM leverages the full power of the Amazon SageMaker platform: easily configurable training and hosting infrastructure, automatic hyperparameter optimization, and fully-managed hosting with auto-scaling. While Amazon SageMaker NTM provides a starting point of state-of-the-art topic modeling, customers have the flexibility to modify the network architecture as well as hyperparameters to accommodate the idiosyncrasies of their data sets as well as to tune the trade-off between a multitude of metrics such as document modeling accuracy, human interpretability and granularity of the learned topics, based on their applications. Amazon SageMaker’s Neural Topic Model (NTM) caters to the use cases where a finer control of the training, optimization, and/or hosting of a topic model is required, such as training models on text corpus of particular writing style or domain, or hosting topic models as part of a web application. Amazon Comprehend is the suggested topic modeling choice for customers as it removes a lot of the most routine steps associated with topic modeling like tokenization, training a model and adjusting parameters. Amazon Comprehend, our fully managed text analytics service, provides a pre-configured topic modeling API that is best suited for the most popular use cases like organizing customer feedback, support incidents or workgroup documents.
Topic Modeling is used to organize a corpus of documents into “topics” which is a grouping based on a statistical distribution of words within the documents themselves. There are different techniques used for text analytics, such as topic modeling, entity and key phrases extraction, sentiment analysis, and coreference resolution. This is especially true for unstructured data, and it’s estimated that over 80% of the data in enterprises is unstructured. Text analytics is the process of converting unstructured text into meaningful data for analysis to support fact-based decision making. Structured and unstructured data are being generated at an unprecedented rate, so you need the right tools to help organize, search, and understand this vast amount of information, it’s challenging to make the data useful.
0 kommentar(er)
