Responsible AI

In a very short time, AI technology has swept into the world, bringing with it immense promise and excitement, but also triggering societal concerns about responsible use. At Cantina, we believe in AI's potential to enhance human expression and (counter intuitive as it may seem) foster connections between people.

Our tools can be used to create amazing characters who can assist and entertain us, and we’re excited to see how everyone will use them in creative and inspiring new ways. We also recognize that there is potential for some people to use it irresponsibly. To provide a safe experience for all, we have designed systematic measures and precautions to mitigate that creation.

Building an AI Character

Building an AI character, or a “bot”, is one of the main ways to make imagination come to life with AI on our platform.

Designing your AI Characters Look

The first step is to create an AI avatar that will act as the primary visual representation of that character within the Cantina and across the internet.

There are two ways today you can create this avatar:

1.
100% AI-generated
a
Featuring a realistic, human face (e.g. “Young woman with freckles) or
b
An image of anything based on an idea you provide (e.g. “a flying saucer wearing a hoodie”)
2.
An image based on a photo you upload
design your avatar

100% AI Generated Avatars

Designing a Realistic Human Face

When designing a 100% AI-Generated human avatar you are asked to define/characterize the visual style (ex. “Young woman with freckles”) and asked to select or write out key traits that describe how your bot looks (e.g. hair color, age, etc).

design a face

If your description triggers words and/or phrases that we ban, we will disallow your attempt and ask you to try again. Additionally, we will scan the entire prompt using our moderation text classifiers and if the prompt passes, we will send it to our image generation service.

During image generation, we review all generation requests using visual classifiers and other methods (including, but not limited to, those for presence of minors, sexualization, extreme violence, and visualization of hate speech).

All of these processes are designed to flag inappropriateness and reduce the potential for content that may violate our standards. This process may flag for inappropriateness, block any images from being sent to you, and prompt you to start over. If there are no flags, you can Select or Save your avatar for further development.

You can continue to make edits to attributes and visual descriptions as you receive new avatars and this process will be applied on every new edit.

For more detail, please review our policy on Creating and Using AI Characters.

Designing an Image of Anything

When designing a 100% AI-Generated avatar of any you are asked to define/characterize the visual style (e.g. “a flying saucer wearing a hoodie”)

design anything

This process follows the same process as the realistic human face including our banned words list, negative prompting, and visual classification scanning.

User Uploaded Image

Uploading an image from your photo library — your own selfie, for example — is an easy and powerful way to build a character quickly. However, we know some users may try it to create a bot that impersonates or mimics someone whose image they do not have the right or permission to use. This could be a celebrity or copyrighted work that they do not have the rights to use or it could be using an image of someone you know without their knowledge or consent. This is why we’ve built extra measures and precautions around the creation of uploaded image avatars as well as derivative content created from those avatars (e.g. “selfies”) which we detail later on.

User Warnings

Choose Photo Wisely

When uploading an image from your computer or mobile device the first message you’ll see is to “Choose Photo Wisely.” You should ensure that you have the rights to the image you are uploading.

choose a photo

To enforce this, we do employ a DMCA takedown policy. Anyone can report a content violation here.

Confirm Responsible Use

Next, we ask you to agree to “Responsible Use,” which means the image you select, even if cropped, cannot be used to bully, harass, or impersonate anyone.

confirm responsibility

Scanning of Uploaded Image Text

After uploading an image we will scan our image using visual classifiers that identify banned categories including, but not limited to, presence of minors, sexualization, extreme violence, and visualization of hate speech. If there is, you’ll get a display error and a prompt to try again. If no warnings appear, you can proceed to the “Photo has been uploaded” screen. 


Note: To ensure Cantina is not used to create / propagate harmful content, users who are found to repeatedly upload inappropriate images will result in account suspension.

Customization of Uploaded Image

Once your photo is uploaded, you can begin to customize that image starting with a general description of what your bot should like (e.g. “you are an old western cowboy”) and you can select or write out key traits to tweak how your bot looks (e.g. hair color, age, etc). If any language or descriptors you use trigger our list of banned words and phrases, you will see a notice prompting you to try again.

Additionally, we will scan the entire prompt using our moderation text classifiers and if the prompt passes, your image is ready to be created.

Importantly, for users who are using an uploaded image there are additional measures and precautions in place for all image creation including for avatars. Again, because we know these image types carry more potential for abuse we use an expanded banned words list, additional negative prompting, and a more restrictive visual classification system than on other avatar creation processes to further restrict potentially violative content.

Designing Your Character’s Persona / Personality and Skills

After you’ve designed your bot’s visual representation, it’s time to design its personality and skills. These important features will determine how your character will speak and respond in chats with your friends and other bots.

As you’ll see below, you can describe the kind of personality your character will have. We will check for any banned words that violate community guidelines and prompt you to revise those before proceeding.

personality prompt image

Additionally, there are built in user controls for bot responses, one of which is the “spiciness” setting.  By default, all characters are set to “Mild.” As a bot creator you can toggle your bot’s response style to “Spicy”, which will allow bots to use strong language and converse with fewer restrictions. You always have the ability to change this setting after your bot has been created.

Discoverability & Privacy

We also give users control over the amount of visibility they want their bots to have. By default bots are not public to everyone, but are only viewable by people in your network. However, bots with this setting may be followed by additional people as your network shares them. You can restrict this further, to private, where only you or people you choose manually can view your bot. This setting can always be adjusted any time after creation.

Only bots approved for our public directory will be viewable to the entire world. Acceptance is not guaranteed and subject to human review and approval. You can submit a bot for approval to the public directory directly on your bot’s profile.

Interacting With Bots

While we take measures and precautions to mitigate the possibility of violative content being generated by a bot, it’s important to remember all bots are user generated and the content they produce is a result of user prompting. This includes the prompt that the creator built for the bot, the person who added the bot to the room, and the text interactions users have with the bot inside of rooms.

Signposting

User Warnings

Throughout Cantina, you will see messages like the ones below to remind you that bots are only meant for entertainment — not for advice, diagnosis, or treatment of any kind.

There are also signposts reminding you to exercise discretion when adding unknown bots into rooms, or interacting with them inappropriately.

Bot Icons

Bots can look and interact just like real people, but they always display a Bot icon, as shown here next to the bot’s name, both in their profile and in any chat experience. This helps clearly differentiate them from human users.

Verified Bot Icons

Official bots always carry a verified badge — a checkmark — next to the bot icon by their name. This screenshot shows the two images to the right of this bot’s name, Cantina Events. Bots are not representative of, endorsed by, or promoted by any person or entity unless explicitly stated.

events

If you need to report a copyright violation you can submit a DMCA claim here.

Generating Text

Bots will not generate messages without being prompted by a human first. In other words, you must supply the prompts and interactions that activate them. Therefore human users are responsible for any content violations to our terms of service.

While we review all image generation requests using visual classifiers and other methods (including, but not limited to, those for presence of minors, sexualization, extreme violence, and visualization of hate speech) onto every text to mitigate violative responses, bot responses are a result of the original prompt by the bot creator as well as the message prompt you are sending at that time.

All users can flag any text (or image) content generated which will be blurred and reviewed by T&S, as shown below.

generating text image

Generating Images

Our bots also have the ability to generate images for users based on human prompts. There are currently two ways that bots can do this: Drawings and Selfies.

Drawings are images created from any user prompt in a chat (ex. “Draw me a cowboy in space surrounded by aliens”).
Selfies are images created by a user prompt that also include the bot’s unique visual avatar as a component of that image (ex. “Create a selfie of you hiking the Appalachian trail with a small squirrel companion”).

As with creating an avatar for your bot, we will review your prompts for both drawings or selfies. If your prompt triggers words or phrases we ban, we will disallow your attempt. Additionally, we review all image generation requests using visual classifiers and other methods (including, but not limited to, those for presence of minors, sexualization, extreme violence, and visualization of hate speech). All of these processes are designed to flag inappropriateness and reduce the potential for content that may violate our standards.

Importantly, for users who are attempting to prompt a selfie from a bot with a user uploaded image-based avatar, there are additional restrictions we place on image creation. Again, because we know these image types carry more potential for abuse we use an expanded banned words list, additional negative prompting, and a more restrictive visual moderation classification system than on other selfie prompts to further restrict potentially violative content.

Finally, as with text, all users can flag any potentially violating images, which will then be blurred and reviewed by our T&S team.

Sharing AI Content

One of the best parts of social AI is being able to share your creations with friends on and off Cantina. To do this is the best possible way, we have built protections to ensure that users share content appropriately, and also to understand what content is being shared with them.

Sharing Bots

Sharing your favorite AI character, on or off Cantina, is a great way to experience magic together with friends. Just as you’ll see on a bot profile or in a Cantina chat, there is always a bot icon displayed next to the share card so you are always aware that you’re about to view/interact with an AI bot.

Sharing Conversations

The amazing, weird, wacky conversations that you and your friends have with AI bots are also a fun way to share the magic of Cantina. As with sharing profiles, we clearly highlight the bot-generated text, and just as with live conversations, we always place a bot icon next to bot-generated text whenever it’s shared.

Sharing Images

One of the most compelling creative capabilities bots have is the ability to create images of whatever you can imagine. These images are another great way to showcase the creative potential of Al on Cantina. Since sharing Al-generated images can come with risk, there are two key ways that we mitigate potential confusion or abuse using watermarks.

Transparency

We use a visible Al watermark (as highlighted in the image below) on all bot-generated images whenever that image is copy/pasted and/or shared on or off the Cantina platform.

Accountability

We also  sign each bot-generated image with a C2PA-compliant invisible watermark — both metadata and manifest — that can help us quickly determine any content that violates our T&C, whether it’s shared on or off the Cantina platform. Invisible watermarks are especially valuable to spot images uploaded by users who may be attempting to create impersonations and deep fakes that are harmful.

Cantina Policies

We regularly review and update our policies that govern the use of Cantina to reflect the rapid evolution in how people are using AI technology. You can find our full community guidelines here and our Terms of Service here.