Creating an UI to add Unsplash images to tldraw

One of the things I miss with tldraw is being able to pull media from outside sources without having to download them or pulling from my filesystem.

In this short tutorial, I’ll teach you how to connect to Unsplash’s API, search photos and drag them into tldraw.

💡 I’m assuming you’re interested in tldraw as a developer platform for building infinite canvas apps. I’m skipping contextualization here but am glad to give more context in another post, let me know on Twitter, or send me an email at meet@hdoro.dev.

What we’ll build ✨

The set-up

I’m assuming you’re working from tldraw-examples and have it set-up. Clone the repository, install its dependencies and run npm run dev, which should open a local server at http://localhost:5173.

Fetching & displaying pictures from Unsplash

Start by registering for an Unsplash developer account, creating a project and generating an API key for it. Then, install the unsplash-js SDK:

npm i --save unsplash-js

# OR

yarn add unsplash-js

We’ll create a React component called UnsplashSidebar.tsx (I’m using Typescript) with a <form> element to take the user’s search query and an internal state to hold the results. It won’t need to touch tldraw, you’ll see why in a minute.

// UnsplashSidebar.tsx
// I didn't include CSS styles to keep the example minimal.
import React from 'react'
import { createApi } from 'unsplash-js'
import { Basic } from 'unsplash-js/dist/methods/photos/types'

export type UnsplashPhoto = Basic

// 1. Create the unsplash SDK
const unsplash = createApi({
	// Using tldraw's example, you can add VITE_UNSPLASH_KEY to your .env file and use it like so 👇
  accessKey: import.meta.env.VITE_UNSPLASH_KEY,
})

// 2. Define a function that gets photos from the API given a user-defined search query
async function runSearch(searchQuery: string) {
  const res = await unsplash.search.getPhotos({ query: searchQuery })
  return res.response?.results
}

function UnsplashSidebar() {
  // 3. Hold the search query in state
  const [searchQuery, setSearchQuery] = React.useState('')
  // And the photos
  const [photos, setPhotos] = React.useState<UnsplashPhoto[]>([])

  // 4. When submitting the form, run the search and update the photos
  async function handleSubmit(e: React.FormEvent<HTMLFormElement>) {
    e.preventDefault()
    const results = await runSearch(searchQuery)
    setPhotos((prevResults) => results || prevResults)
  }

  return (
    <div>
      {/* 5. render the form */}
      <form onSubmit={handleSubmit}>
        <input
          type="search"
          value={searchQuery}
          onChange={(e) => setSearchQuery(e.target.value)}
        />
        <button type="submit">Search</button>
      </form>

      {photos.map((photo) => (
        <figure
          key={photo.id}
        >
          <img
            src={photo?.urls?.small}
            alt={photo.alt_description || ''}
          />
          <cite>
            By {photo.user?.name} (
            <a
              href={photo.links?.html}
              target="_blank"
              rel="noopener noreferrer"
            >
              source
            </a>
            )
          </cite>
          <figcaption>{photo.description}</figcaption>
        </figure>
      ))}
    </div>
  )
}

export default UnsplashSidebar

💡 To explore: I recommend exploring the UnsplashPhoto type to see what cool things you can do with their pictures.

With this, you should be able to fetch and display pictures from Unsplash, but how do we add them to tldraw when users drag them into the canvas?

The solution is using HTML’s drag events and deferring the tldraw logic to a centralized drop handler in the canvas. First, we say pictures are draggable:

<figure
  // ...other props
  draggable
  onDragStart={(event) => handlePictureDragStart(event, photo)}
>

And attach their data to the drag event when users start dragging. Here’s handlePictureDragStart:

const handlePictureDragStart = React.useCallback(
  (event: React.DragEvent<HTMLElement>, picture: UnsplashPhoto) => {
    event.dataTransfer.setData('unsplash-photo', JSON.stringify(picture))
  },
  [],
)

Now, our tldraw canvas can acess the dragEvent.dataTransfer.getData('unsplash-photo') to inform itself of what is being dragged onto the screen. Starting from a simplified version of CustomUiExample:

import {
  Canvas,
  ContextMenu,
  TldrawEditor,
  TldrawUi,
  TldrawUiContextProvider,
  useApp,
} from '@tldraw/tldraw'
import '@tldraw/tldraw/editor.css'
import { addImageToCanvas } from './addImageToCanvas'
import UnsplashSidebar, { UnsplashPhoto } from './UnsplashSidebar'

export default function CustomUiExample() {
  return (
    <div className="tldraw__editor">
      <TldrawEditor>
        <TldrawUiContextProvider>
          <div>
            <TldrawUi />

            {/* 1. Add the sidebar */}
            <UnsplashSidebar />

            <ContextMenu>
              {/* 2. And a custom canvas */}
              <CustomCanvas />
            </ContextMenu>
          </div>
        </TldrawUiContextProvider>
      </TldrawEditor>
    </div>
  )
}

// 3. Set up the canvas to accept drops from the sidebar
function CustomCanvas() {
  const app = useApp()

  // 4. Function handle dropped unsplash photos
  function addUnsplashPhoto(
    photo: UnsplashPhoto,
    event: React.DragEvent<Element>,
  ) {
    addImageToCanvas({
      app,
      image: {
        src: photo.urls.full,
        h: photo.height,
        w: photo.width,
        mimeType: `image/${new URL(photo.urls.full).searchParams.get('fm')}`,
        name:
          photo.description ||
          photo.alt_description ||
          `Unsplash photo by ${photo.user.name}`,
        id: photo.id,
      },
      event,
    })
  }

  return (
    <Canvas
      // 5. Connect to tldraw by overriding its onDrop handler, only when there's an unsplash photo
      **onDropOverride={(defaultDrop) => async (event) => {
        const unsplashPhoto = event.dataTransfer?.getData('unsplash-photo')
        if (unsplashPhoto) {
          try {
            const photo = JSON.parse(unsplashPhoto) as UnsplashPhoto
            return addUnsplashPhoto(photo, event)
          } catch (error) {}
        }

				// If any other dropped item, default to tldraw's handler
        defaultDrop(event)
      }}**
    />
  )
}

From the above, the most important bit is our usage of the onDropOverride API, where we do our own special case handling when the item dropped is an Unsplash photo. Remember the handlePictureDragStart above? That’s where it’s being used.

Finally, we need to define the addImageToCanvas helper, which can be used beyond Unsplash (think adding images from Giphy, Google Photos, Iconify, etc.):

import { App } from '@tldraw/tldraw'

export function addImageToCanvas({
  app,
  image,
  event,
}: {
  app: App
  image: {
    id: string
    src: string
    h: number
    w: number
    mimeType: string
    name?: string
  }
  event: React.DragEvent<Element>
}) {
  const shapeId = app.createShapeId(`${image.id}_${Date.now()}`)
  const assetId = `asset:${image.id}`

  // 1. If there's no coresponding asset for the image in tldraw's store, create one
  if (!app.getAssetById(assetId)) {
    app.createAssets([
      {
        id: assetId,
        type: 'image',
        typeName: 'asset',
        props: {
          src: image.src,
          h: image.h,
          w: image.w,
          isAnimated: false,
          mimeType: image.mimeType,
          name: image.name || image.src,
        },
      },
    ])
  }

  // 2. Ensure the image is scaled to fit the available size in the viewport
  const dimensions = scaleToFit(
    image.w,
    image.h,
    app.viewport.w * 0.6,
    app.viewport.h * 0.6,
  )

  // 3. Convert the mouse position from screen coordinates to tldraw's page coordinates
  const mousePosition = app.screenToPage(event.clientX, event.clientY)

  // 4. Create the shape of the image
  app.createShapes([
    {
      id: shapeId,
      type: 'image',
      x: mousePosition.x - dimensions.w / 2,
      y: mousePosition.y - dimensions.h / 2,
      props: {
        assetId: assetId,
        h: dimensions.h,
        w: dimensions.w,
      },
    },
  ])

  // 5. Finally, select the shape so the user gets immediate feedback
  app.select(shapeId)
}

function scaleToFit(w: number, h: number, maxW: number, maxH: number) {
  const ratio = Math.min(maxW / w, maxH / h)

  return { w: w * ratio, h: h * ratio }
}

This may seem like a lot of code, but when we consider we’re getting an amazing canvas that handles all sorts of weird edge cases (especially arrows!), it feels absurdly empowering!

Reach out if you’re building with infinite canvases, it’d be a pleasure to chat 🙂