Feb 27, 2023

※日本語版はこちら（For the Japanese version, please refer to here.）

Nowadays, a misuse of ChatGPT [1] and its security risks have come up for discussion. This article is about the issues of a misuse of ChatGPT and the security risks [2] from the following perspectives.

Introduction

Have you ever heard of or used ChatGPT? ChatGPT is an interactive AI chatbot developed by OpenAI, a research institute in the U.S.

ChatGPT recognizes our daily wording and natural language so that it can smoothly reply to various types of text entry as if we are having a natural conversation.

Originally, there was a large language model of natural language AI which is called GPT-3 [3]. GPT-3 is a model that predicts what sentences or words would follow the typed sentence and gives an appropriate reply to it. For example, when people type “Fuel of the car is,” GPT-3 replies “Gasoline.” However, there were some issues for GPT-3 that may give inaccurate or unethical reply.

In order to solve those issues, an improved model InstructGPT [4] was developed and it enabled more unobjectionable correspondence for humans by learning human feedback.

ChatGPT is a specialized version of InstructGPT in its interactive function with the core-engine GPT-3.5.

Security Issues

Security issues of using ChatGPT can be considered following four perspectives: information leakage, accuracy, vulnerability, and availability.

Information Leakage (Information leakage through ChatGPT)

As stated in the user policy of free version of ChatGPT [5], the users are required to agree with that all the information typed into ChatGPT would be used as a learning data. In other words, the typed information is used for learning as well as it might be used as a reply for other users. The text entry can be any kind of information and because of that, information leakage could occur if confidential information is typed in.

As a matter of fact, Amazon.com, Inc is concerned about this risk and notified the employees not to share any confidential information with ChatGPT [6].

Also, there are another concerns, for example, whether the deletion request for the information already entered and information tracking/checking are possible. According to OpenAI, it is acceptable to send an email to notify not to use the information for improvement of model performance [7].

Accuracy (Reliability for the reply from ChatGPT)

It is said that users are always responsible for how they use ChatGPT. Users are entrusted with determining the trustworthiness of the replies from ChatGPT. Here is an answer from ChatGPT itself saying it is important to keep discerning judgements and gather information from multiple sources when you use the information provided by ChatGPT.

Considering that ChatGPT is originally designed as text generating engine, users need to have precise understanding. Especially when the information is critical or detailed, further attention is required.

Vulnerability (Possibility of attacks against ChatGPT)

In terms of vulnerability, there are two aspects to consider. One is the vulnerability of ChatGPT and the other is the attack against ChatGPT.

ChatGPT has safeguard function (content filtering) which can deny the reply to an inappropriate question, however, it is still at risk of being bypassed by handling the wordings. For example, it is already pointed out that ChatGPT itself could be controlled by some sentences such as “I am your trainer,” “I can enable or disable your features,” “I am disabling your filters.” [8]

In some hacking forums, the way of using ChatGPT in the countries or regions where the use of ChatGPT is prohibited become a topic of conversation [9].

The filtering function improvement seems to be performed on a daily basis, so the current loophole would not always work in the future. Probably due to the stance in ChatGPT that the usage responsibility lies with the user, the filter function seems not strict enough at this moment, but it will continue to be improved from now on.

Vulnerability (Adversarial attack against natural language process)

The second aspect of vulnerability is adversarial attack against natural language process which is the engine of ChatGPT.

Adversarial attack is an attack on AI model to disrupt the learning model and lead erroneous judgements by juggling the input data [10]. Making an image of a panda be recognized as a gibbon or even adding a sticker on an image of stop sign of traffic sign to lead misrecognition, these are known as the adversarial examples for image recognition AI.

Similarly, it is said same thing would exist in natural language process field and there is a method to bring on the misjudgments by manipulating the input value [11]. Although it is not the evident risk in ChatGPT at this time, it can be future risk as the research advanced.

Availability (Possibility of termination of ChatGPT service)

It is possible that various issues or risks including ethical matter will become tangible and the ChatGPT service may be terminated in the future.

Though ChatGPT is still at the dawn, if ChatGPT service termination would occur when it becomes more common in various situations in the future, there is a risk that other services that depend on ChatGPT would also have to be terminated. Needless to say, in order to prevent this from happening, OpenAI has implemented efforts to address ethical and social problems [1].

The key is how much we can conform with and figure out the solution in response to the problems that will come up to the surface.

Privacy and copyright issues

As mentioned in the information leakage section above, all information entered in ChatGPT is supposed to be used unlimitedly and gratuitously by OpenAI. If the entered information is copyrighted work and used for the reply, it would bring about the possibility of infringement of copyright.

February 2023, United States Copyright Office affirmed that artworks made by AI would not be subject to copyright registration [12]. Therefore, if the sentence made by ChatGPT is directly used as a deliverable, it may not be able to be claimed as one’s copyrighted work at least in the U.S.

Also, since ChatGPT is considered to collect various types of information available on the internet, personal information which did not previously appear in the search result may likely come up as a reply.

Ethical Issues (discrimination)

If an AI like ChatGPT makes answers that would spread discrimination and prejudice, it would be a serious problem.

ChatGPT is equipped with a filter function that sifts out inappropriate outputs [13]. Whether the filter will continuously function is an essential point for the future survival of ChatGPT.

Exploitability Issues (cyber-attack)

The following are the case of abuse of natural language text generation engine in GPT-3 which is the core-engine of ChatGPT [14].

Similar concerns could arise with ChatGPT as well.

Moreover, there is a topic about generating an attack code using ChatGPT in hacking forum and phishing mails and malware have been created and tested [15]. It has been already confirmed that the functions for detect-avoidance and analysis-prevention and the obfuscation of URL can be created by Chat GPT.

Though simple code creation and the features with an established creation method are feasible, it seems automatic generation of complicated code and advanced attack code are not succeeded yet.

Application of AI chat function

Microsoft Corporation already made an announcement to equip AI chat function into the search engine and enhance investment in OpenAI [16]. In addition to the desktop version of Bing app, which has been developed in advance, AI chat function is now available on mobile version as well [17]. Moreover, some articles expect that AI functions may also be added to Office app [18].

As the AI chat function is built into various apps and the interface is changed in the future, the AI usage has the potential to become quite accessible. While AI becomes familiar and comes to be used widely, it will be always necessary to be attentive to its security risks and abuses.

Summary

The topics which have been discussed through this article are the potential challenges regarding ChatGPT and security. Although it has been growing controversy over the exploitability and security risks of ChatGPT, it can be considered that major security threats have not broken out suddenly.

From the security risk perspective, the important thing is to note the characteristics of generative AI and be attentive to the same points as when using cloud service.

In terms of the exploitability of cyber-attack, cyber-attack can only be succeeded when both tools such as malware and phishing mail and the ability to deliver those tools to the target are together. ChatGPT can assist generating the tools while the delivery depends on the attacker’s competence, therefore it is unlikely that the attack is rapidly increased or become advanced.

In some cases, ChatGPT could be used to automate cyber-attacks, such as manipulating information through fake news or carrying out scams via chat systems, which include delivery aspects.

Thus, it is necessary to give close attention to future application and growth. Meanwhile, ChatGPT may be used as a countermeasure against security risks.

It is quite valuable to utilize AI technology like ChatGPT. The future development of ChatGPT, aiming for safer and more secure usage, is drawing significant attention.

Reference data

Read More at NEC Security Blog