DALL-E¶

This notebook shows how to use OpenAI's DALL-E image API endpoints.

There are three API endpoints:

• Generations: generates an image or images based on an input caption
• Edits: edits or extends an existing image
• Variations: generates variations of an input image

Setup¶

• Import the packages you'll need
• Import your OpenAI API key: You can do this by running ``export OPENAI_API_KEY="your API key"\ in your terminal.
• Set a directory to save images to

In [1]:

# imports
import openai  # OpenAI Python library to make API calls
import requests  # used to download images
import os  # used to access filepaths
from PIL import Image  # used to print and edit images

# set API key
openai.api_key = os.environ.get("OPENAI_API_KEY")

# imports import openai # OpenAI Python library to make API calls import requests # used to download images import os # used to access filepaths from PIL import Image # used to print and edit images # set API key openai.api_key = os.environ.get("OPENAI_API_KEY")

In [2]:

# set a directory to save DALL-E images to
image_dir_name = "images"
image_dir = os.path.join(os.curdir, image_dir_name)

# create the directory if it doesn't yet exist
if not os.path.isdir(image_dir):
    os.mkdir(image_dir)

# print the directory to save to
print(f"{image_dir=}")

# set a directory to save DALL-E images to image_dir_name = "images" image_dir = os.path.join(os.curdir, image_dir_name) # create the directory if it doesn't yet exist if not os.path.isdir(image_dir): os.mkdir(image_dir) # print the directory to save to print(f"{image_dir=}")

image_dir='./images'

Generations¶

The generation API endpoint creates an image based on a text prompt.

Required inputs:

• prompt (str): A text description of the desired image(s). The maximum length is 1000 characters.

Optional inputs:

• n (int): The number of images to generate. Must be between 1 and 10. Defaults to 1.
• size (str): The size of the generated images. Must be one of "256x256", "512x512", or "1024x1024". Smaller images are faster. Defaults to "1024x1024".
• response_format (str): The format in which the generated images are returned. Must be one of "url" or "b64_json". Defaults to "url".
• user (str): A unique identifier representing your end-user, which will help OpenAI to monitor and detect abuse. Learn more.

In [3]:

# create an image

# set the prompt
prompt = "A cyberpunk monkey hacker dreaming of a beautiful bunch of bananas, digital art"

# call the OpenAI API
generation_response = openai.Image.create(
    prompt=prompt,
    n=1,
    size="1024x1024",
    response_format="url",
)

# print response
print(generation_response)

# create an image # set the prompt prompt = "A cyberpunk monkey hacker dreaming of a beautiful bunch of bananas, digital art" # call the OpenAI API generation_response = openai.Image.create( prompt=prompt, n=1, size="1024x1024", response_format="url", ) # print response print(generation_response)

{
  "created": 1667611641,
  "data": [
    {
      "url": "https://oaidalleapiprodscus.blob.core.windows.net/private/org-l89177bnhkme4a44292n5r3j/user-dS3DiwfhpogyYlat6i42W0QF/img-SFJhix3AV4bmPFvqYRJDkssp.png?st=2022-11-05T00%3A27%3A21Z&se=2022-11-05T02%3A27%3A21Z&sp=r&sv=2021-08-06&sr=b&rscd=inline&rsct=image/png&skoid=6aaadede-4fb3-4698-a8f6-684d7786b067&sktid=a48cca56-e6da-484e-a814-9c849652bcb3&skt=2022-11-05T01%3A27%3A21Z&ske=2022-11-06T01%3A27%3A21Z&sks=b&skv=2021-08-06&sig=0ZHl38v5UTFjA7V5Oshu8M58uHI5itEfvo2PX0aO6kA%3D"
    }
  ]
}

Note: If you get this error - AttributeError: module 'openai' has no attribute 'Image' - you'll need to upgrade your OpenAI package to the latest version. You can do this by running pip install openai --upgrade in your terminal.

In [4]:

# save the image
generated_image_name = "generated_image.png"  # any name you like; the filetype should be .png
generated_image_filepath = os.path.join(image_dir, generated_image_name)
generated_image_url = generation_response["data"][0]["url"]  # extract image URL from response
generated_image = requests.get(generated_image_url).content  # download the image

with open(generated_image_filepath, "wb") as image_file:
    image_file.write(generated_image)  # write the image to the file

# save the image generated_image_name = "generated_image.png" # any name you like; the filetype should be .png generated_image_filepath = os.path.join(image_dir, generated_image_name) generated_image_url = generation_response["data"][0]["url"] # extract image URL from response generated_image = requests.get(generated_image_url).content # download the image with open(generated_image_filepath, "wb") as image_file: image_file.write(generated_image) # write the image to the file

In [5]:

# print the image
print(generated_image_filepath)
display(Image.open(generated_image_filepath))

# print the image print(generated_image_filepath) display(Image.open(generated_image_filepath))

./images/generated_image.png

Variations¶

The variations endpoint generates new images (variations) similar to an input image.

Here we'll generate variations of the image generated above.

Required inputs:

• image (str): The image to use as the basis for the variation(s). Must be a valid PNG file, less than 4MB, and square.

Optional inputs:

• n (int): The number of images to generate. Must be between 1 and 10. Defaults to 1.
• size (str): The size of the generated images. Must be one of "256x256", "512x512", or "1024x1024". Smaller images are faster. Defaults to "1024x1024".
• response_format (str): The format in which the generated images are returned. Must be one of "url" or "b64_json". Defaults to "url".
• user (str): A unique identifier representing your end-user, which will help OpenAI to monitor and detect abuse. Learn more.

In [6]:

# create variations

# call the OpenAI API, using `create_variation` rather than `create`
variation_response = openai.Image.create_variation(
    image=generated_image,  # generated_image is the image generated above
    n=2,
    size="1024x1024",
    response_format="url",
)

# print response
print(variation_response)

# create variations # call the OpenAI API, using `create_variation` rather than `create` variation_response = openai.Image.create_variation( image=generated_image, # generated_image is the image generated above n=2, size="1024x1024", response_format="url", ) # print response print(variation_response)

{
  "created": 1667611666,
  "data": [
    {
      "url": "https://oaidalleapiprodscus.blob.core.windows.net/private/org-l89177bnhkme4a44292n5r3j/user-dS3DiwfhpogyYlat6i42W0QF/img-7HTTBl2k9l4Ir4BTHXnJvFz9.png?st=2022-11-05T00%3A27%3A46Z&se=2022-11-05T02%3A27%3A46Z&sp=r&sv=2021-08-06&sr=b&rscd=inline&rsct=image/png&skoid=6aaadede-4fb3-4698-a8f6-684d7786b067&sktid=a48cca56-e6da-484e-a814-9c849652bcb3&skt=2022-11-04T01%3A50%3A22Z&ske=2022-11-05T01%3A50%3A22Z&sks=b&skv=2021-08-06&sig=QlcKhn427bOAQobM8CmpEf3K90OiumP5jOQwkJpcH6Y%3D"
    },
    {
      "url": "https://oaidalleapiprodscus.blob.core.windows.net/private/org-l89177bnhkme4a44292n5r3j/user-dS3DiwfhpogyYlat6i42W0QF/img-KGKrKGzlsXN0INxaeII2t8XG.png?st=2022-11-05T00%3A27%3A46Z&se=2022-11-05T02%3A27%3A46Z&sp=r&sv=2021-08-06&sr=b&rscd=inline&rsct=image/png&skoid=6aaadede-4fb3-4698-a8f6-684d7786b067&sktid=a48cca56-e6da-484e-a814-9c849652bcb3&skt=2022-11-04T01%3A50%3A22Z&ske=2022-11-05T01%3A50%3A22Z&sks=b&skv=2021-08-06&sig=RbPoAwXMVfdPxKF40ZjVjlclrnzaQZS%2BxzhgkEcYhOk%3D"
    }
  ]
}

In [7]:

# save the images
variation_urls = [datum["url"] for datum in variation_response["data"]]  # extract URLs
variation_images = [requests.get(url).content for url in variation_urls]  # download images
variation_image_names = [f"variation_image_{i}.png" for i in range(len(variation_images))]  # create names
variation_image_filepaths = [os.path.join(image_dir, name) for name in variation_image_names]  # create filepaths
for image, filepath in zip(variation_images, variation_image_filepaths):  # loop through the variations
    with open(filepath, "wb") as image_file:  # open the file
        image_file.write(image)  # write the image to the file

# save the images variation_urls = [datum["url"] for datum in variation_response["data"]] # extract URLs variation_images = [requests.get(url).content for url in variation_urls] # download images variation_image_names = [f"variation_image_{i}.png" for i in range(len(variation_images))] # create names variation_image_filepaths = [os.path.join(image_dir, name) for name in variation_image_names] # create filepaths for image, filepath in zip(variation_images, variation_image_filepaths): # loop through the variations with open(filepath, "wb") as image_file: # open the file image_file.write(image) # write the image to the file

In [8]:

# print the original image
print(generated_image_filepath)
display(Image.open(generated_image_filepath))

# print the new variations
for variation_image_filepaths in variation_image_filepaths:
    print(variation_image_filepaths)
    display(Image.open(variation_image_filepaths))

# print the original image print(generated_image_filepath) display(Image.open(generated_image_filepath)) # print the new variations for variation_image_filepaths in variation_image_filepaths: print(variation_image_filepaths) display(Image.open(variation_image_filepaths))

./images/generated_image.png

./images/variation_image_0.png

./images/variation_image_1.png

Edits¶

The edit endpoint uses DALL-E to generate a specified portion of an existing image. Three inputs are needed: the image to edit, a mask specifying the portion to be regenerated, and a prompt describing the desired image.

Required inputs:

• image (str): The image to edit. Must be a valid PNG file, less than 4MB, and square.
• mask (str): An additional image whose fully transparent areas (e.g. where alpha is zero) indicate where image should be edited. Must be a valid PNG file, less than 4MB, and have the same dimensions as image.
• prompt (str): A text description of the desired image(s). The maximum length is 1000 characters.

Optional inputs:

• n (int): The number of images to generate. Must be between 1 and 10. Defaults to 1.
• size (str): The size of the generated images. Must be one of "256x256", "512x512", or "1024x1024". Smaller images are faster. Defaults to "1024x1024".
• response_format (str): The format in which the generated images are returned. Must be one of "url" or "b64_json". Defaults to "url".
• user (str): A unique identifier representing your end-user, which will help OpenAI to monitor and detect abuse. Learn more.

Set Edit Area¶

An edit requires a "mask" to specify which portion of the image to regenerate. Any pixel with an alpha of 0 (transparent) will be regenerated. The code below creates a 1024x1024 mask where the bottom half is transparent.

In [9]:

# create a mask
width = 1024
height = 1024
mask = Image.new("RGBA", (width, height), (0, 0, 0, 1))  # create an opaque image mask

# set the bottom half to be transparent
for x in range(width):
    for y in range(height // 2, height):  # only loop over the bottom half of the mask
        # set alpha (A) to zero to turn pixel transparent
        alpha = 0
        mask.putpixel((x, y), (0, 0, 0, alpha))

# save the mask
mask_name = "bottom_half_mask.png"
mask_filepath = os.path.join(image_dir, mask_name)
mask.save(mask_filepath)

# create a mask width = 1024 height = 1024 mask = Image.new("RGBA", (width, height), (0, 0, 0, 1)) # create an opaque image mask # set the bottom half to be transparent for x in range(width): for y in range(height // 2, height): # only loop over the bottom half of the mask # set alpha (A) to zero to turn pixel transparent alpha = 0 mask.putpixel((x, y), (0, 0, 0, alpha)) # save the mask mask_name = "bottom_half_mask.png" mask_filepath = os.path.join(image_dir, mask_name) mask.save(mask_filepath)

Perform Edit¶

Now we supply our image, caption and mask to the API to get 5 examples of edits to our image

In [10]:

# edit an image

# call the OpenAI API
edit_response = openai.Image.create_edit(
    image=open(generated_image_filepath, "rb"),  # from the generation section
    mask=open(mask_filepath, "rb"),  # from right above
    prompt=prompt,  # from the generation section
    n=1,
    size="1024x1024",
    response_format="url",
)

# print response
print(edit_response)

# edit an image # call the OpenAI API edit_response = openai.Image.create_edit( image=open(generated_image_filepath, "rb"), # from the generation section mask=open(mask_filepath, "rb"), # from right above prompt=prompt, # from the generation section n=1, size="1024x1024", response_format="url", ) # print response print(edit_response)

{
  "created": 1667611683,
  "data": [
    {
      "url": "https://oaidalleapiprodscus.blob.core.windows.net/private/org-l89177bnhkme4a44292n5r3j/user-dS3DiwfhpogyYlat6i42W0QF/img-F5XQFFBLrN7LdXuG5CkQJpxr.png?st=2022-11-05T00%3A28%3A03Z&se=2022-11-05T02%3A28%3A03Z&sp=r&sv=2021-08-06&sr=b&rscd=inline&rsct=image/png&skoid=6aaadede-4fb3-4698-a8f6-684d7786b067&sktid=a48cca56-e6da-484e-a814-9c849652bcb3&skt=2022-11-04T02%3A06%3A29Z&ske=2022-11-05T02%3A06%3A29Z&sks=b&skv=2021-08-06&sig=2UhH%2BkKdvDVoRcgWJhmNFVzpvLzBAZpnA/tU80Zc8M0%3D"
    }
  ]
}

In [11]:

# save the image
edited_image_name = "edited_image.png"  # any name you like; the filetype should be .png
edited_image_filepath = os.path.join(image_dir, edited_image_name)
edited_image_url = edit_response["data"][0]["url"]  # extract image URL from response
edited_image = requests.get(edited_image_url).content  # download the image

with open(edited_image_filepath, "wb") as image_file:
    image_file.write(edited_image)  # write the image to the file

# save the image edited_image_name = "edited_image.png" # any name you like; the filetype should be .png edited_image_filepath = os.path.join(image_dir, edited_image_name) edited_image_url = edit_response["data"][0]["url"] # extract image URL from response edited_image = requests.get(edited_image_url).content # download the image with open(edited_image_filepath, "wb") as image_file: image_file.write(edited_image) # write the image to the file

In [12]:

# print the original image
print(generated_image_filepath)
display(Image.open(generated_image_filepath))

# print edited image
print(edited_image_filepath)
display(Image.open(edited_image_filepath))

# print the original image print(generated_image_filepath) display(Image.open(generated_image_filepath)) # print edited image print(edited_image_filepath) display(Image.open(edited_image_filepath))

./images/generated_image.png

./images/edited_image.png