Generative AI
artificial intelligence that generates plausible content
generative AI usually structured or unstructured data?
unstructured data
... [Show More] (e.g. images, audio, texts)
Brainpower
Read More
Previous
Play
Next
Rewind 10 seconds
Move forward 10 seconds
Unmute
0:02
/
0:15
Full screen
How does Generative AI learn the pattern?
learn the underlying pattern from training data
What is generative AI used for?
generating similar content or solving downstream tasks
What does GitHub Copilot do?
Based on GPT-3; gives autocomplete code suggestions to programmers
Examples of downstream tasks
Completion, Summarization
GPT3
large language model (LLM) with 175B parameters
LLM
large language model
Who is GPT-3 monetized
pay per query basis
What is the impact of the consumer facing side doing to GPT-3?
crashing the server
What is Sudowrite
Built on GPT3 - writing software
What is Viable?
CRM softerm-5tware using GPT-3 to automatically summarize customer feedback for business
What level of GPT does ChatGPT utilize?
Built off GPT-4, it technically is referred to as GPT3.5
What makes a large language model "large"
it is NOT the data set used for training; it is because of the 175B parameters (very computationally intensive)
2 most common Training opensource Datasets
CommonCrawl; Webtext (repository for online pages); Online book Corpora; Wikipedia
What is GPT3
developer facing version providing access to LLM via an API (pick some parameters to adjust output)
What is the "temperature" of the model?
i.e. change the temperature of the model --> change the temperature of the output changes (i.e. specifying non-fiction, fiction)
What does finetuning do?
trains a pretrained model on a new dataset without training from scratch; "transfer learning," produce more accurate models with less training time; i.e. not changing all 175B parameters; customite it to you rneeds
What is transfer learnings
small adjustments / finetuning
What happens as the levels of GTP go up?
more data
Is the purpose of GPT to answer questions correctly?
No, purpose is to fill the text (completion, generating similar content)
Limitations of GPT in a math problem
purpose isn't to answer question correctly; predictively guessing next level
Generative AI is supervised or unsupervised learning?
supervised (take a set of words in your data set, the words that follow is what you're trying to predict)
Limitations of ChatGP
doesn't solve problems well; factually incorrect answers; biases; training is historic (Oct 2021); expensive to retrain; trained in batch mode; lack of transparency around sources; plagiarism
Markov assumptions
probability of next word depending solely on a limited number of preceding words
Bi-gram
predicting words only conditioning on the previous word
Tri-gram
conditioning on the previous 2 words
N-gram
Previous N words
once you use Bi-gram, Tri-gram, and N-gram data, what happens?
probability can be computed from training data
Why isn't the N-gram model perfect?
curse of dimensionality (number of possible 10-word sequences in English); sparsity issue (combinations of words not found in data set)
What is a solution to the limitations of the N-gram model?
deep neural networks (deep learning)
solutions to deepfakes
regulations, detection technology, Microsoft's AMP (Authentication of Media via Provenance) which enables media content creators to create and assign a certificate of authenticity to their content
Why is regulation not sufficient to solve deepfake isseus?
regulation is slow to catch up [Show Less]