-2
\$\begingroup\$

Is it possible to use chatGPT or any other similar technology to simply describe what you want photoshop to do (perform OCR on all text in an image generated by Dall-E 3 and replace it with a font and text that is spell checked and correct". It's failry straightforward to find a guide on any process in the adobe help files, why is this feature of automation missing (or a form of interactive automation, where necessery information is asked for).

For instance, if i want to know how to do anything with a smart object it's easy to find the well written and complete guides online in the adobe help file system. It basically provides me with step-by-step instructions to get the task done.

In addition, photoshop has advanced capabilities for recognizing objects easily and in context.

Now GPT-v can analyze images it seems to me that a system in which everything can be done through prompts in combination with a prompt generator specifically written for graphic design or adobe photoshop.

For instance, Dall-e 3 cartoons still have major text spelling problems.

A prompt to this 'Photoshop Prompt System' could be...

'fix text in this image using a similar font and use the image description to infer what the text should be, if it is too warped'

a prompt generator could then generate a number of steps,

  1. OCR all text in the image.

  2. Find the font through a OpenAI plugin or Photoshop Plugin that is similar, and install it in photoshop (seperate number of steps).

  3. Analyze how the text is aligned, the size etc.

  4. try to spell check the text, if still unintelligible, try to reconstruct what it could have said from the description that accompanies a Dall-E 3 image (supposing this is an OpenAI Dall-E 3 generated cartoon)

  5. If not possible, prompt the user that text regeneration was not un-ambigiously possible and that the user should provide the text directly or perhaps recreate the cartoon (or have photoshop change the cartoon to by generating a whole new series of steps to modify the image (which might be a bit too ambitious).

  6. If all text is extracted from the image, select the wrong original text and use the remove features to remove this text from the background. (using photoshop's neural filter technology).

  7. Replace the text with the proper text, similar font, alignment and size.

Now i understand this involves several layers of using GPT-v, GPT, openai plugins, photoshop features etc. But for many tasks this code itself could be generated partly by AI as well.

I could not find anything about such a direction of photoshop or a OpenAI (or other technology based product) that could provide this automation.

The move from traditional photo editing/retouching/designing to generative fills etc. especially in after effects where exciting for a moment. Now i just feel overwhelmed by a workflow of an application that seems to not want (?) to structurally connect their guides and help files and process them through LLM's and AI technology to allow photoshop to become highly automated and retire the necessity for users to be intimately knowledgeable about layers, linking, smart objects and other concepts that were very exciting pre nov-2019 i guess...

The only reason i can think of that they transition this slowly is to not alienate the professional photoshop users, or make them redundant for a large number of tasks. The reality of course is, however jarring that may be for some, is that it will happen eventually - whether it's adobe or any other technology that will provide this last 'bridge' to automating everything from screen output, to mouse and keyboard input.

Have I missed something? Any ideas?

\$\endgroup\$
3
  • \$\begingroup\$ If you think this is so easy, I suggest you write a plugin to do it. \$\endgroup\$
    – Philip Kendall
    Nov 17, 2023 at 3:39
  • \$\begingroup\$ Get text and font recognized involve send image to AI engine. And image/text may be subject of copyright. Currently AI engines do not have very clear rules what they do with text/images they receive. So I, personally will not use such tool (if/when exist). \$\endgroup\$ Nov 17, 2023 at 7:03
  • \$\begingroup\$ AI-generated stuff looks cool and is even right in some cases, but so far it is not right every time. AI-based tools are OK if you are going to accept whatever they return. They aren't adequate if you know exactly what you want. Adjusting a skin tone by pushing a cursor is still a lot faster that explaining an AI what precise tint you want. \$\endgroup\$
    – xenoid
    Nov 17, 2023 at 13:39

1 Answer 1

2
\$\begingroup\$

Not sure if this is an answer or a comment.

But are you aware that Adobe has lowly been rolling out generative AI in its products under the FireFly banner?

Adobe has already rolled this technology out to multiple products (including Photoshop), and has a roadmap to rolling it out to all its products.

So the first steps have already been taken, but the technology is relatively new. Remember that Dall-E is less than 3 years old, and ChatGPT is less than 1 year old. Considering the advancements made in this time, I'm sure that Adobe will be doing a lot of experimentation and development in the future.


I recently went to a demonstration of FireFly and came away excited as to what it could do. When I got home I tried using it on a photo I took in order to remove some chairs from the scene. I managed to remove about 1/2 of the chairs in the scene. For another 1/4, even though I said "delete", it actually replaced my chairs with different chairs. And for the final 1/4 it actually refused to do anything, saying that my edits were prohibited by community guidelines. Researching the error message suggested that Adobe thought I was creating lewd content. And that is troubling for multiple reasons.

\$\endgroup\$
2
  • \$\begingroup\$ I was part of beta testing of FireFly and see serious progress. But there are a lot of areas and place for improvement. \$\endgroup\$ Nov 17, 2023 at 14:33
  • \$\begingroup\$ @RomeoNinov I agree the future is rushing towards us, and we're still not there yet. But I bristle at restrictions due to "community guidelines". Up until that point I was thinking about dropping $$ on Adobe, after that, "hell, no" \$\endgroup\$
    – Peter M
    Nov 17, 2023 at 14:40

Not the answer you're looking for? Browse other questions tagged or ask your own question.