GPT-4 vs. ChatGPT: An Exploration of Training, Performance, Capabilities, and Limitations | by Mary Newhauser | Mar, 2023

GPT-4 is an enchancment, however mood your expectations.

OpenAI surprised the world when it dropped ChatGPT in late 2022. The brand new generative language mannequin is anticipated to completely remodel total industries, together with media, schooling, legislation, and tech. Briefly, ChatGPT threatens to disrupt nearly all the things. And even earlier than we had time to really envision a post-ChatGPT world, OpenAI dropped GPT-4.

In current months, the velocity with which groundbreaking massive language fashions have been launched is astonishing. Should you nonetheless don’t perceive how ChatGPT differs from GPT-3, not to mention GPT-4, I don’t blame you.

On this article, we are going to cowl the important thing similarities and variations between ChatGPT and GPT-4, together with their coaching strategies, efficiency and capabilities, and limitations.

ChatGPT and GPT-4 each stand on the shoulders of giants, constructing on earlier variations of GPT fashions whereas including enhancements to mannequin structure, using extra refined coaching strategies, and growing the variety of coaching parameters.

Each fashions are based mostly on the transformer structure, which makes use of an encoder to course of enter sequences and a decoder to generate output sequences. The encoder and decoder are related by an consideration mechanism, which permits the decoder to pay extra consideration to essentially the most significant enter sequences.

OpenAI’s GPT-4 Technical Report provides little data on GPT-4’s mannequin structure and coaching course of, citing the “aggressive panorama and the protection implications of large-scale fashions.” What we do know is that ChatGPT and GPT-4 are in all probability educated in the same method, which is a departure from coaching strategies used for GPT-2 and GPT-3. We all know rather more concerning the coaching strategies for ChatGPT than GPT-4, so we’ll begin there.

ChatGPT

To start out with, ChatGPT is educated on dialogue datasets, together with demonstration knowledge, wherein human annotators present demonstrations of the anticipated output of a chatbot assistant in response to particular prompts. This knowledge is used to fine-tune GPT3.5 with supervised studying, producing a coverage mannequin, which is used to generate a number of responses when fed prompts. Human annotators then rank which of the responses for a given immediate produced the perfect outcomes, which is used to coach a reward mannequin. The reward mannequin is then used to iteratively fine-tune the coverage mannequin utilizing reinforcement studying.

To sum it up in a single sentence, ChatGPT is educated utilizing Reinforcement Learning from Human Feedback (RLHF), a method of incorporating human suggestions to enhance a language mannequin throughout coaching. This enables the mannequin’s output to align to the duty requested by the consumer, reasonably than simply predict the following phrase in a sentence based mostly on a corpus of generic coaching knowledge, like GPT-3.

GPT-4

OpenAI has but to reveal particulars on the way it educated GPT-4. Their Technical Report doesn’t embrace “particulars concerning the structure (together with mannequin measurement), {hardware}, coaching compute, dataset development, coaching methodology, or related.” What we do know is that GPT-4 is a transformer-style generative multimodal mannequin educated on each publicly accessible knowledge and licensed third-party knowledge and subsequently fine-tuned utilizing RLHF. Apparently, OpenAI did share particulars relating to their upgraded RLHF strategies to make the mannequin responses extra correct and fewer more likely to veer outdoors security guardrails.

After coaching a coverage mannequin (as with ChatGPT), RLHF is utilized in adversarial coaching, a course of that trains a mannequin on malicious examples supposed to deceive the mannequin with a view to defend the mannequin towards such examples sooner or later. Within the case of GPT-4, human area specialists throughout a number of fields charge the responses of the coverage mannequin to adversarial prompts. These responses are then used to coach further reward fashions that iteratively fine-tune the coverage mannequin, leading to a mannequin that’s much less possible to provide out harmful, evasive, or inaccurate responses.

Capabilities

When it comes to capabilities, ChatGPT and GPT-4 are extra related than they’re completely different. Like its predecessor, GPT-4 additionally interacts in a conversational fashion that goals to align with the consumer. As you may see under, the responses between the 2 fashions for a broad query are very related.

OpenAI agrees that the excellence between the fashions might be delicate and claims that “distinction comes out when the complexity of the duty reaches a adequate threshold.” Given the six months of adversarial coaching the GPT-4 base mannequin underwent in its post-training section, that is in all probability an correct characterization.

In contrast to ChatGPT, which accepts solely textual content, GPT-4 accepts prompts composed of each pictures and textual content, returning textual responses. As of the publishing of this text, sadly, the capability for utilizing picture inputs isn’t but accessible to the general public.

Efficiency

As referenced earlier, OpenAI experiences vital enchancment in security efficiency for GPT-4, in comparison with GPT-3.5 (from which ChatGPT was fine-tuned). Nevertheless, whether or not the discount in responses to requests for disallowed content material, discount in poisonous content material technology, and improved responses to delicate matters are as a result of GPT-4 mannequin itself or the extra adversarial testing is unclear presently.

Moreover, GPT-4 outperforms CPT-3.5 on most tutorial {and professional} exams taken by people. Notably, GPT-4 scores within the ninetieth percentile on the Uniform Bar Examination in comparison with GPT-3.5, which scores within the tenth percentile. GPT-4 additionally considerably outperforms its predecessor on conventional language mannequin benchmarks in addition to different SOTA fashions (though generally simply barely).

Each ChatGPT and GPT-4 have vital limitations and dangers. The GPT-4 System Card consists of insights from an in depth exploration of such dangers performed by OpenAI.

These are only a few of the dangers related to each fashions:

• Hallucination (the tendency to provide nonsensical or factually inaccurate content material)
• Producing dangerous content material that violates OpenAI’s insurance policies (e.g. hate speech, incitements to violence)
• Amplifying and perpetuating stereotypes of marginalized individuals
• Producing practical disinformation supposed to deceive

Whereas ChatGPT and GPT-4 battle with the identical limitations and dangers, OpenAI has made particular efforts, together with in depth adversarial testing, to mitigate them for GPT-4. Whereas that is encouraging, the GPT-4 System Card finally demonstrates how susceptible ChatGPT was (and probably nonetheless is). For a extra detailed rationalization of dangerous unintended penalties, I like to recommend studying the GPT-4 System Card, which begins on web page 38 of the GPT-4 Technical Report.

On this article, we overview an important similarities and variations between ChatGPT and GPT-4, together with their coaching strategies, efficiency and capabilities, and limitations and dangers.

Whereas we all know a lot much less concerning the mannequin structure and coaching strategies behind GPT-4, it seems to be a refined model of ChatGPT that now accepts picture and textual content inputs and claims to be safer, extra correct, and extra inventive. Sadly, we must take OpenAI’s phrase for it, as GPT-4 is just accessible as a part of the ChatGPT Plus subscription.

The desk under illustrates an important similarities and variations between ChatGPT and GPT-4:

The race for creating essentially the most correct and dynamic massive language fashions has reached breakneck velocity, with the discharge of ChatGPT and GPT-4 inside mere months of one another. Staying knowledgeable on the developments, dangers, and limitations of those fashions is crucial as we navigate this thrilling however quickly evolving panorama of enormous language fashions.