1. Amazon Comprehend• Natural Language Processing and Text Analytics• Input social media, emails, web pages, documents, transcripts, medical records (Comprehend Medical)• Extract:– Key phrases, – Entities, – Sentiment, – Language, – Syntax, – Topics, – Document classifications• Can train on your own data1.1. Entities
1.2. Key Phrases
1.3. Language
1.4. Sentiment
1.5. Syntax
2. Amazon Translate• Uses deep learning for translation– Supports custom terminology– In CSV or TMXformat– Appropriate for proper names, brand names, etc.
3. Amazon Transcribe• Speech to text– Input in FLAC, MP3, MP4, or WAV, in a specified language– Streaming audio supported (HTTP/2 or WebSocket)* French, English, Spanish only• Speaker Identificiation– Specify number of speakers• Channel Identification– i.e., two callers could be transcribed separately– Merging based on timing of “utterances”• Automatic Language Identification– You don’t have to specify a language; it can detect the dominant one spoken.• Custom Vocabularies– Vocabulary Lists (just a list of special words – names, acronyms)– Vocabulary Tables (can include “SoundsLike”, “IPA”, and “DisplayAs”)
4. Amazon Polly• Neural Text-To-Speech, many voices & languages• Lexicons– Customize pronunciation of specific words & phrases– Example: “World Wide Web Consortium” instead of “W3C”• SSML– Alternative to plain text– Speech Synthesis Markup Language– Gives control over emphasis, pronunciation, breathing, whispering, speech rate, pitch, pauses.• Speech Marks– Can encode when sentence / word starts and ends in the audio stream– Useful for lip-synching animation
5. Amazon Rekognition• Computer vision• Object and scene detection– Can use your own face collection• Image moderation• Facial analysis• Celebrity recognition• Face comparison• Text in image• Video analysis– Objects / people / celebrities marked on timeline– People Pathing
5.1. Rekognition: The Nitty Gritty• Images come from S3, or provide image bytes as part of request– S3 will be faster if the image is already there• Facial recognition depends on good lighting, angle, visibility of eyes, resolution• Video must come from Kinesis Video Streams– H.264 encoded– 5-30 FPS– Favor resolution over framerate• Can use with Lambda to trigger image analysis upon upload
5.2. New in 2020: Rekognition Custom Labels• Train with a small set of labeled images• Use your own labels for unique items• Example: the NFL (National Football League in the US) uses custom labels to identify team logos, pylons, and foam fingers in images.
6. Amazon Forecast• Fully-managed service to deliver highly accurate forecasts with ML• “AutoML” chooses best model for your time series data– ARIMA, DeepAR, ETS, NPTS, CNN-QR Prophet• Works with any time series– Price, promotions, economic performance, etc.– Can combine with associated data to find relationships• Inventory planning, financial planning, resource planning• Based on “dataset groups,” “predictors,” and “forecasts.”
6.1. More on Forecast algorithms• CNN-QR– Convolutional Neural Network – Quantile Regression– Best for large datasets with hundreds of time series– Accepts related historical time series data & metadata• DeepAR+– Recurrent Neural Network– Best for large datasets– Accepts related forward-looking time series & metadata• Prophet– Additive model with non-linear trends and seasonality• NPTS– Non-Parametric Time Series– Good for sparse data. Has variants for seasonal / climatological forecasts• ARIMA– Autoregressive Integrated Moving Average– Commonly used for simple datasets (<100 time series)• ETS– Exponential Smoothing– Commonly used for simple datasets (<100 time series)
7. Amazon Lex• Billed as the inner workings of Alexa• Natural-language chatbot engine• A Bot is built around Intents– Utterances invoke intents (“I want to order a pizza”)– Lambda functions are invoked to fulfill the intent– Slots specify extra information needed by the intent* Pizza size, toppings, crust type, when to deliver, etc.• Can deploy to AWS Mobile SDK, Facebook Messenger, Slack, and Twilio7.1. Amazon Lex Automated Chatbot Designer• You provide existing conversation transcripts• Lex applies NLP & deep learning, removing overlaps & ambiguity• Intents, user requests, phrases, values for slots are extracted• Ensures intents are well defined and separated• Integrates with Amazon Connect transcripts
8. Amazon Personalize• Fully-managed recommender engine– Same one Amazon uses• API access– Feed in data (purchases, ratings, impressions, cart adds, catalog, user demographics etc.) via S3 or API integration– You provide an explicit schema in Avro format– Javascript or SDK– GetRecommendations* Recommended products, content, etc.* Similar items– GetPersonalizedRanking* Rank a list of items provided* Allows editorial control / curation• Console and CLI too8.1. Amazon Personalize Features• Real-time or batch recommendations• Recommendations for new users and new items (the cold start problem)• Contextual recommendations– Device type, time, etc.• Similar items• Unstructured text input• Intelligent user segmentation– For marketing campaigns
8.2. Amazon Personalize Terminology• Datasets– Users, Items, Interactions• Recipes– USER_PERSONALIZATION– PERSONALIZED_RANKING– RELATED_ITEMS• Solutions– Trains the model– Optimizes for relevance as well as your additional objectives* Video length, price, etc. – must be numeric– Hyperparameter Optimization (HPO)• Campaigns– Deploys your “solution version”– Deploys capacity for generating real-time recommendations
8.3. Amazon Personalize Hyperparameters• User-Personalization, Personalized-Ranking– hidden_dimension (HPO)– bptt (back-propagation through time - RNN)– recency_mask (weights recent events)– min/max_user_history_length_percentile (filter out robots)– exploration_weight 0-1, controls relevance– exploration_item_age_cut_off – how far back in time you go• Similar-items– item_id_hidden_dimension (HPO)– item_metadata_hidden_dimension (HPO with min & max range specified)8.4. Maintaining Relevance• Keep your datasets current– Incremental data import• Use PutEvents operation to feed in real-time user behavior• Retrain the model– They call this a new solution version– Updates every 2 hours by default– Should do a full retrain (trainingMode=FULL) weekly
8.5. Amazon Personalize Security• Data not shared across accounts• Data may be encrypted with KMS• Data may be encrypted at rest in your region (SSE-S3)• Data in transit between your account and Amazon’s internal systems encrypted with TLS 1.2• Access control via IAM• Data in S3 must have appropriate bucket policy for Amazon Personalize to process it• Monitoring & logging via CloudWatch and CloudTrail8.6. Amazon Personalize Pricing• Data ingestion: per-GB• Training: per training-hour• Inference: per TPS-hour• Batch recommendations: per user or per item
9. Other ML Services• Amazon Textract– OCR with forms, fields, tables support• AWS DeepRacer– Reinforcement learning powered 1/18scale race car• DeepLens– Deep learning-enabled video camera– Integrated with Rekognition, SageMaker, Polly, Tensorflow, MXNet, Caffe– 9.1. Industrial Applications• Amazon Lookout– Equipment, metrics, vision– Detects abnormalities from sensor data automatically to detect equipment issues– Monitors metrics from S3, RDS, Redshift, 3 rd party SaaS apps– Vision uses computer vision to detect defects in silicon wafers, circuit boards, etc.• Amazon Monitron– End to end system for monitoring industrial equipment & predictive maintenance
10. TorchServe, AWS Neuron, AWS Panorama• TorchServe– Model serving framework for PyTorch– Part of the PyTorch open source project from Facebook (Meta?)• AWS Neuron– SDK for ML inference specifically on AWS Inferentia chips– EC2 Inf1 instance type– Integrated with SageMaker or whatever else you want (deep learning AMI’s, containers, Tensorflow, PyTorch, MXNet)– 10.1. AWS Panorama• Computer Vision at the edge• Brings computer vision to your existing IP cameras
11. Deep Composer, Fraud Detection, CodeGuru, Contact Lens• AI-powered keyboard• Composes a melody into an entire song• For educational purposes•
11.1. Amazon Fraud Detector• Upload your own historical fraud data• Builds custom models from a template you choose• Exposes an API for your online application• Assess risk from:– New accounts– Guest checkout– “Try before you buy” abuse– Online payments11.2. Amazon CodeGuru• Automated code reviews!• Finds lines of code that hurt performance• Resource leaks, race conditions• Offers specific recommendations• Powered by ML• Supports Java and Python
11.3. Contact Lens for Amazon Connect• For customer support call centers• Ingests audio data from recorded calls• Allows search on calls / chats• Sentiment analysis• Find “utterances” that correlate with successful calls• Categorize calls automatically• Measure talk speed and interruptions• Theme detection:discovers emerging issues11.4. Amazon Kendra• Enterprise search with natural language– For example, “Where is the IT support desk?” “How do I connect to my VPN?”• Combines data from file systems, SharePoint, intranet, sharing services (JDBC, S3) into one searchable repository• ML-powered (of course) – uses thumbs up / down feedback• Relevance tuning – boost strength of document freshness, view counts, etc.• Alexa’s sister? I don’t know, but that’s one way to remember it•
12. Amazon Kendra, Amazon Augmented AI (A2I)• Human review of ML predictions• Builds workflows for reviewing low-confidence predictions• Access the Mechanical Turk workforce or vendors• Integrated into Amazon Textract and Rekognition• Integrates with SageMaker• Very similar to Ground Truth
