P3: Survey Data and Session Summary

Survey Responses

Interview Transcript

P3: The first tool I probably like ChatGPT if we're thinking AI tools I guess, it was just to make funny images really you know. It hadn't really become this thing in work

Paul: What other AI tools since then have you adopted? What have you maybe left by the wayside and abandoned?

P3: So at work we're a Microsoft shop, so we have co-pilot. Figma Make we just got, not a big fan of it.

00:03:28

Paul: I've heard that before. Why? Why is that?

P3: It looks like under the hood it's pulling in some like it almost feels like it's an iFrame that's injecting either half of what you get out of Google AI studio or the other half is like the Anthropic pieces. The token limit, like the deal we have right now the token limit has been for some people restricting it just like I like to work first with like a data model like you know you obviously you have your regular design process you have a problem or something you're going after but like even to get a data into the system or at least the way our IT has it locked down it's a plugin and they don't approve the plugin. So I'm like it's not useful where I'm at to just generate a bunch of random screens. You know we work for a bank so it's like we have a design system we know what our screens are going to be.

00:04:24

P3: I mean I created a couple useful tools, but it's not it's not it hasn't been my jam. Like Google AI Studio has been my like favorite tool to use. , that's my primary tool, but it's all side of desk. I literally when we when this came out, they didn't even roll it out to everybody. They just kind of started staggering it and I'm like, I got to learn this. So, I would sit here and then my actual AI computer was on my left. So,

Paul: So that was off book. You said that was your personal use?

P3: Yeah. personal. And then just to learn it not so while waiting for them to say we're going to roll it out.

P3: I was afraid we went through a lot of hiring and firing. So yeah, those are my current like jam tools I guess.

Paul: What else have you tried and then stopped using? And why.

00:05:05

P3: I haven't really I mean outside of I was really impressed with Google AI Studio. It's basically take Gemini, but then go imagine that Gemini is the outside of the car and then getting into the car and actually being able to set controls and have a full access to all their different model types has been very helpful. You can actually build things with it and then it's a direct pipeline to some of their other systems. So if you wanted to scale it up, they have the Google Cloud infrastructure, they have other pieces if you wanted to make it secure. So, it felt like a more closer to what you would need to get something to be in the market versus other tools. , back in the day,

P3: I used to use it wasn't AI, but I used to use a lot of Wii whissywig tools to build games. , so it felt a little bit familiar to that,

P3: But yeah, I haven't tried the other ones.

00:06:05

P3: I'm just like, this is working for me right now. It's not cost prohibitive. I do pay for a tier one license, but I haven't hit my cap yet, which is fine.

Paul: Did I understand you correctly that you are a Microsoft shop so you've got mandated Copilot, but you're not allowed to use Google AI Studio you've got a manual step of getting your ideation and design and concepts from your personal to your work, right?

P3: Yeah, you can't use any [non-approved applications] they lock. There's no Google access at all at our work. So, the personal stuff I have not shared with work.

Paul: I thought you were saying that you were using the Google AI Studio for some stuff that you just can't do with the tools you're given at work, but that's not the case.

00:06:53

P3: I mean I am, a couple of the things that I have demoed like for example we recently decided that our application is all going to be native Spanish which was fine project they did that but when then when we realized our design system was not made for this many characters so at the time we had some nearshore developer or nearshore designers who were native speaking Spanish speakers long story short it turned into a kind of a giant manual process of having a team in real time go through the mobile screens, assess where the UI was breaking, come up on the spot with like how do you fix this and I was like this is not scalable and we lost those workers. So side of desk I built a tool that lets you inject UI elements translated into there's like six languages but I just basically for a Spanish see where it breaks write the code to correct the logic for how you would want it to adapt when it breaks like this and then export that as a JSON file and a view logic for the developers to see how you did that deal.

00:08:05

P3: I've shared the idea with the bank in terms of connecting my iPad and say this is something I want to pursue [but] it's still like we have this giant pipeline. So that one I've shared the other ones I'm like I don't know I don't want I don't want to like our unfortunate reality of a lot of [financial services companies] or at least the one I'm in is patents are looked at as a defense. So there's like a huge patent mandate every year to hit a certain number and very few make it to market. And I'm like some of the things I've built I'm like I really want these to get out and market to help people and I don't know that I can rely on my current employer to do that.

P3: The other tools I've built within the bank but it's constricted like copilot is not a build tool. I mean it's good for like some of the design management stuff we used but it's not you can't build with it.

00:08:54

P3: Figma Make you can kind of build with but again it's like trying to drive somebody else's car so you know I'm sure maybe if you play with but then I the cap is pretty low on the credits so I'm like "crap I didn't want to burn through these credits now I'm in stuck in the water" and then you can't publish so I'm like okay and then you can't do some of the things where like you can upload a file set but it's not like I can create my own environment and say okay here's the data I want you to pull from you can't get it a set of like an ice who can give us system instructions. So for this tool I'm building, we're going to start with these system instructions, use these all the time as like the guiding principle, then with the prompts of the individual kind of things we're trying to do. You can't do that. So it's like it's like Figma, you're just making us burn through credits and then it I've I've watched and I don't know why it does this like three or four prompts later.

00:09:46

P3: It's like it forgot the prompt you gave it and the result has changed and you're like now I'm burning minutes for something that you just did three steps ago that was great. So it's like this weird kind of like what how specific do I have to be in this? It was a little frustrating in that regard.

Paul: wWhat do you think your biggest win or success or efficiency gain has been with using AI?

P3: One was really like when we got our new COO the mandate came down like all these areas were running too hot in terms of budget and we were heavily reliant on third party vendors. So we had two weeks we lost all the vendors like all of them like 45% reduction but just in the design team. So we were still supporting 14 delivery pods who had not lost their developers POS BAS and we're like oh crap how are we going to keep the ship right?

00:10:42

P3: So the first thing we did is was kind of a combination like it we realized that the teams in the old model weren't very organized in how they were operating because they had dare I say the luxury to not be organized. What I mean by that is they were just assigned to a delivery pod. So like I just work on whatever assigned to me. I get it done. I get it done. Nobody was managing where the time was going. nobody was doing any kind of like active capture of work so that everything that was coming in was in one spot. So we had a sense of volume. Long story short, we came in and did kind of like a you know like a what's his face David Allen five steps to getting stuff done process over 30 days where AI came in is we started, kind of capture first you have to limit all your capture buckets then your capture you have to where you

00:11:29

P3: went in first we well we used the first part we used AI too is just let's run analysis on your current calendars like where's your time going because nobody was tracking it everybody's like well I'm working for a pie okay but on But how much is meetings? And then what we found out is some of the teams are like 40 to 60% booked on just meetings. I'm like, well, when are you getting your work done? Like, well, we are. And I'm like, well, then your capacity is like, what are we looking at like eight hours a week? That's not So, we did that first AI got that and then said, okay, we're not going to do this. We pulled the team out of agile, the whole design team in my group, and said we're we won, we don't have the people to do. We don't have enough people to just evenly divide you amongst 14 teams. So, we're going to be one centralized team and then we're going to pull in all the intake from all these groups.

00:12:17

P3: We're going to triage that intake. assign it to designers and then your calendar is going to be booked by meetings obviously focus time, dedicated focus time, people time, your lunch and your breaks, and then professional development. And then we're going to in the calendar using AI. Each one of those would get a category and a little kind of a tick mark. So it' say focus time tick mark and then what and then what we found is you could then take co-pilot and run cues against that batch cues against it to get your results of like your team members are spending how much time in meetings versus focus time and we're looking to get and what we've we've been able to get is a healthy balance of that. then we stopped accepting sight of desk request. So all we created it wasn't with AI but we created forms for our intake and then we use AI then to go triage the forms to see what's new let us know when things are in how much do we have what's our typical time for completing a task and a review so we can get a realistic s to the business partners and say okay if we're going to do this work so and then we used it to AI to then generate weekly reports of we're still stable

00:13:33

P3: As soon as we yes see that we've got too much capacity or not enough capacity, our attack is off that our capacity is full but the demand, we can flag that right away. We can actually start to like we call it wedding DJ is kind of like book people out and everybody's been trained to book their calendar like a DJ and say I know I have item A. I'm going to spend my next three focus buckets on that. And it's my focus bucket. And when it's the rolls, your head's down. You don't accept meetings during that time, you know, because otherwise it's like where does the work go? like it's going to you can't get it done. So, it's it's we've been able to stay afloat. We haven't got through our first PI increment yet.

P3: So, this will be our first test of supporting both in-flight delivery and the pre-work up front. But, it's been super helpful to at least for the design op side and that's really its capability.

00:14:23

Paul: This sounds like you guys were faced with a design ops challenge that driven not just by AI but the layoffs and the you know the need and then you rose to it.

Paul: What's been the biggest win as far as the organization as a whole? Not just design.

00:16:58

Paul: What's been your biggest disappointment or failure that you've experienced while trying to implement or use AI tools?

P3: Yeah. It's so I have like a lovehate relationship with it. So on one end there's no like a royalty model. So we know that a lot of the models were trained on other people's intellectual property and we know that the com if it's a generative model that it's it's a composite of other people's things.

P3: They don't get there's no compensation for it. So, even like I just I think I put out today I'm trying to find more

P3: Illustrators to pay to do like even just the stuff for like a blog because I'm like I don't want that to go away and if it becomes a commodity where anybody can write a prompt and get art then there will be no real I mean you do art for art's sake but you can't make a career out of it if nobody's willing to pay for it.

00:18:07

P3: So that part has been disappointing. it worked that the hardest part was like the way the roll out happened, you know, you got a couple courses in co-pilot and it's like just go play with it. Well, it's like it's like the big miss there was like no,

P3: We actually need people to go take some fundamental courses on learning how these models and things work. And then even then it's like when you get the model and even if they change like you one you don't know what version you're on, two when it gets updated you're like what changed? how come the prompt I was using with the same data set yesterday does not respond the same now. So, that's been disappointing. This kind of like you know something's happening, but you don't actually see what ingredients are going into the cake, if you will. , and then just simple things like the it depending on because it's all tied into the quoteunquote

00:18:54

P3: microverse. Depending on your level of access in the company, you can really be hamstrung. Like I tried to do something where was simple as hey I need to schedule 15 one-on-ones with these our vendors that we're let go. Here's their email addresses. It's like yeah, I can do that. It goes off 10 minutes later. Can't because it doesn't have calendar access. Nor can it go get calendar access. I'm like I just, you know, you're going back and forth. It's like if you could just tell me up front like what is your capability and not say you can do something you can't would be great.

00:19:45

Paul: Is there anything that you're doing really differently now because you've implemented AI?

P3: Yeah. So first every day I ask myself when I do something is this something I can either automate or enhance and not do manually. So case in point is every quarter these different product groups will do product demos and they all have still have to even though it's a digital product they still want to do PowerPoint decks to send to executives and we get all these requests of like hey can you take this screenshot or you can take this thing and put it into this iPhone template which is not a difficult task if you have Figma but they don't have Figma so now they're asking us to do it and we've we're drastically reduced our staff so I was like I bet I could just build this. So that's what I did. I built a tool that lets you first upload your whatever your stock image is. And I pulled all the ones we had licenses for. And as long as the area you want to implement the screen is a solid color, you upload that first. Then it has kind of like you remember Paintshop Pro or Photoshop back in the day with the bucket tool. That's how it determines the mask. So you just draw.

Paul: So you built this as a self-service tool for other people.

P3: At the bank because I don't want to do certain things anymore.

Paul: Okay. So, yeah. Yeah. You don't want to be the screenshot

P3: Yeah. Yeah.

Paul: monkey.

P3: Exactly.

00:21:18

P3: And it's just like now you guys can get repetitive. We're saving time and being able to do the kind of work I want to do in terms of product innovation and not to say it's a commodity, but like the tools are there now. So, and it was it was a cool little use case of like okay like the hardest part about doing that is the fingers, right? People have a finger on the screen. Well, the paint shop bucket thing does it does it really well. It just fills the area. Then that area you upload the image you want there and then you have your standardized rotation controls and up and down to skew the image however you want to match the and export and you're done.

Paul: How how about things that you've started doing that you wouldn't have done before AI was available?

00:22:01

P3: So I'm trying to get an like a true executive position. So it's like one of those things like my work is if you if you if you subscribe to like the belief that like if the work you have to pick to do is not only the work but it has to be the right work for the enterprise and your decision is going to determine the outcome on that then you're working as an executive whether you are or not entitled. So part of my goal is to get into that upper like I realized there was I don't even know what to call it like you know how they say for females there's a glass ceiling. there's a some sort of material ceiling between the your workforce and the executives. So like at our ind our companies 100,000 people 2% are executives but that's where all the decisions are being made right. So, and then it's like for me if we look at design as like all the activities you do to identify potential opportunities to hitting targets in market that's a long custody chain and where design is employed at our company is not where it needs to be to actually drive the types of things I'm trying to see.

00:23:07

P3: so what I'm getting at is in order to do that I started interviewing executives at the

P3: bank and then taking AI like for one example like I was like how does this executive like it always felt like they were reading a script like every presentation was very like polished and I was like how do they have the time? Then I found out they have a whole team that writes for them. So then I started taking those newsletters and corporate communications and feeding them into co-pilot and then had co-pilot build me how to write like this executive. What are their key points? How do they say their things? And then started using that as my own guide to say anytime I answer an email. Write your draft first. Then plug in that upload that thing and say I want to speak and write like an executive. compare what I wrote and give me a draft version of that is more on in line with how they communicate. and then even that taking that same data set and then saying there's what the executives want the high just points in get me to the actions and there's the other level of like we want the deep stories take the same set of data and create three different kind of newsletters for different audiences based on the same set very quickly and then I have it scheduled so each week I'm pulling my reports from planner I'm pulling my calendars across my teams to get all that information helping to create a better one coherent story and

00:24:31

P3: two making the invisible work that we were just doing visible because I think we part of the reason we had such a slash was nobody was seeing that invisible work. So I think AI has really helped with that.

Paul: Yeah, that's a great use case that you've applied it to.

Paul: Let me give you a situation where everyone has probably experienced this either professionally or personally or both where someone shares something and it just it's it seems low effort low quality AI slop.

00:25:33

P3: Boom.

Paul: Do you see norms and unwritten rules forming about AI use at work and AI and disclosure of I use AI for this are you seeing unwritten rules and norms evolving and being created in real time or is it a lot more implicit than that? What's your thought what's your thoughts on that?

P3: That's a great question. It feels a lot It feels similar to like not necessarily I don't know how old you are. like I'm going to be 50 this year, but like I look at other generations that have kind of a digital identity, a highly curated digital identity, and I think to myself, is that who you are in person? Like, it's a filter. It's a you can take the shot a bunch of different times.

P3: You know, you don't have the double chin. I feel like there's going to be or already is this kind of split with when somebody's writing AI, how much are you letting the AI drive?

00:26:37

P3: And then when you speak, how well does that match you?

P3: And are you are you projecting wisdom, but you don't have the experience or the wisdom? And then when you get called on the on the field without that AI,

P3: are you going to be able to speak to it? which is kind of like I think fundamentally there's an issue two I think I think people need to know but it's I don't know at what how to do it in a way like right now there's some giveaways right like people like the long M dash or whatever okay ai wrote that or if you know the personally and you're like that's not how you speak at all then you start to get these hints I don't know that's a great one like I feel like in for art I mean there's been a couple cases that have come up recently, right? Where these AI fake bands have spun up Spotify, used AI to generate listens and made a lot of money.

00:27:34

P3: So then there's been AI art and music and like you know, like why are you taking the human thing from us?

P3: But somebody's creating that. Yeah. I really wish there was some sort of labeling on like it was like food ingredient. How much?

Paul: No artificial ingredients or intelligence.

P3: That's why I was asking for the for those illustrators. I was like, I just want I don't want any AI art. I like want what you're creating because that process of creating that and having your own style is derivative of your unique experiences in life and you can't get that from AI. And then if we don't support, it's going to go away. which would stink.

00:28:25

P3: I don't know if we're going to get the kind of regulation like they have maybe in Europe on it. , it's I don't know. It's it's tricky, too, because then, you know, look, you look at companies and they're like saying, "Hey, you need to use this." And then, but then at the same time, like, you can't cheat during the interview, but people are cheating during the interview. , I don't know. I the my biggest fear is that we're not replacing the apprentice level people like and they still need a fundamental of whatever their craft is without

00:29:17

P3: AI and then augmented with AI if you what happens when we retire who's going to like Plato said who's going to watch the watchers who will know that something is wrong because they never did it you know it's going to be that's

Paul: How this is making you feel, this AI transformation but rather than say how does it make you feel what do you think is the most significant breakthrough or positive outcome that AI might enable in the next decade so first positive and then we'll get to the negative apocalypse.

P3: Yeah, the positive is the barrier for creating things that are pretty realistic or at least have the ability to get you to the next spot is drastically lowered.

00:30:20

P3: Like I think about like when I was making video games, I found a platform in and I did like 12 games in a in a year, but it was still like very limited to the capabilities of that platform versus now that's kind of opening up. So you I've already seen it like a friend of mine has already written a Photoshop clone that just has the top 20 things you would want in it. He's going to sell it because he's tired of playing subscription. people are there's going to be a like this huge explosion of bespoke software that's going to take on the giants and say why do we need this everybody can do this thing or decommoditize it I think that's the advantage like if you know how to prompt correctly or you get at least a fundamental baseline on something you could then have a really good start to go meet with a patent attorney say is there something patentable here and then get a technical expert to look under the hood and say is this scalable what's missing etc in the past you'd be stuck at whatever the tech stack was and your proficiency in it or having to rely on another technical expert and not being able to ask quoteunquote an expert, AI expert, are they bullshitting me? Is this what should I be asking them?

P3: I that's not my hill of gravel. I've never climbed that hill. I can't tell if they're doing this well, you know? yeah,

P3: I think that's exciting. It's also this is going to sound really weird but like when we shifted from I sound like a vampire saying this from the industrial revolution to thought work like 30s 40s we're at another inflection point. So the question is like we used to be able to carry our knowledge with us and that was our com that was our skill set we could parlay for for work that gets democratized. How do you protect yourself?

P3: Do you have to now make an AI version of yourself that you lease or get royalties on when somebody engages with you? Do you have to scale yourself that way?

00:32:20

P3: That's what it's got me thinking about. Like same time I'm working at work, I'm like, am I inadvertently working myself out of a job? And then if that's the case, then what am I going to make that I can commoditize to stay afloat and not become a farmer because I'm terrible at farming. , but you know what I mean? Like I mean I my I joke with my wife all the time.

P3: I know we don't need this many people anymore. we could automate all this and look how much money we could save and slashing cost before they realize, you know, it's like the whole Home Depot, you can remodel your home.

00:33:13

P3: No, you're not an expert at that. Go ahead, get your card and try. You're probably going to have to pay an expert to come back and fix what you broke.