A quick structure for piloting a change

A recent correspondent asked how to move his team from “thoughts and fears about plans” to action, and I suggestion making their next intervention a pilot. That way an intervention is an experiment, and can be evaluated, then continued, tweaked, or ended.

I counseled him to consider coaching folks to articulate

  • the problem we see
  • the result we want
  • what we’ll try, for how long
  • how we will know if it is working or not
  • how we will know if we should stop the pilot early

At the end of the pilot you can evaluate the results and decide to

  • continue as-is
  • continue with tweaks
  • try something else
  • stop altogether

with the same five points above for any change or new intervention.

Mathilda asks: Can you all share how you distinguish product designers from user experience designers?

Mathilda asks:

Can you all share how you distinguish product designers from user experience designers? I’ve been trying to determine the differentiation with other UX friends, but it still seems a bit foggy. Some have explained it to me as user experience designers focus on users and usability, and product designers focus on “everything”, i.e. the product and the business. Many of the user experience books and resources I read (Lean UX, Build Better Products, UX Strategy, NNgroup) though seem to frequently connect business outcomes with UX. I’ve also heard the difference explained as product designers having greater visual design specialization, but I’ve seen that in UX designer roles as well. Also, why not hire a visual designer in that situation?

Mathilda in Where are The Black Designers? Slack

It IS foggy, and as usual, I think the term “product designer” was coined to clarify but failed to do so (and introduced conflict with industrial designers among others). I think it mainly comes from poor mutual understanding of the term “UX” and what all a UX designer does or might do in different settings. This varies WIDELY by organization; in general, the larger the org the narrower the role of an individual contributor UX designer and the more the job(s) of working on a specific experience are distributed over multiple people. This narrowing of the role of a UX designer in large organizations has led to folks who do more than “Edward wifreframe-makin’-hands” to adopt the “Product Designer” term.

Here’s what I see as a rough breakdown:

  • Graphic: the visual design of things, especially print pieces, marketing collateral, informational websites
  • UI: the visual and microinteractive design of web applications, mobile applications, and desktop applications
  • UX: the interaction design of web applications, mobile applications, desktop applications, and occasionally embedded or physical interfaces, with an emphasis on arranging the larger requirements and workflows to meet business goals and user needs together, and the research required to do a good job of this (sometimes some of this UX role is split off into a separate Researcher role)
  • Product: UX + UI

But companies don’t necessarily follow this, and these terms mean different things to different people, and so it’s always useful to

  • when looking for a job
    • be open to multiple titles in your job search
    • look for clues in the job descriptions you are considering
    • explain your capabilities and aptitudes rather than trying to choose a title – use plain English
    • ask questions of your prospective employers about what they mean when they say they want a {whatever} designer and compare their answers to what you want to do (always a good idea anyhow)
  • when hiring
    • be thoughtful and consistent with the language you use in your organization
    • explain what you’re really looking for when you write a job description rather than relying on the title
    • eliminate needless qualifications and job requirements from the job description
    • write a job ad that describes daily and regular activities and how these contribute to org success
    • remember that a job description is not a job ad
  • when talking within your org
    • be thoughtful and consistent with the language you use in your organization
    • coach managers and leaders to be consistent in this same way, which will require explaining why
    • plan needed capacity and capabilities before settling on roles and titles

What would you recommend to someone who is interested in starting with coding/designing/managing, but doesn’t know exactly where to start?

How do I get experience doing a thing without a job doing the thing? By doing the thing anyway.

For design or coding there are two good places to start, and you probably should pursue both:

  1. How can adding a bit of design or coding enhance your current job? Are there repetitive tasks that might be automated, information that could be brought together into a dashboard, metrics or research that might inform your work or the work of your team, places where quality might be improved through greater understanding or thoughtfulness? You can offer to do things, or just start to do them. You can learn a lot by applying new skills to something you already know about.
  2. What parts of coding or design can you try with what you already have, on your own time? I got started in design because I found desktop publishing tools fun to play with in college, and used them to make greeting cards, tee shirts, and to enhance my classwork. Playing with the tools in ways that scratched my own itches taught me skills that I could later apply to my work.

In re point 2 above, my daughter mentioned to me that she might be interested in filmmaking. I told her, “you have a phone, start making some films!” Getting started can be that easy. Your work might not be won’t be very good at first. That’s fine. Keep going unless you find you don’t like the process. Competence will come later.

Starting in management is a little different, but again I see two straightforward paths and it might be worthwhile to pursue both:

  1. Offer to help/take ownership of small moments in your job where coordination is needed, process change is needed, or a problem needs to be sorted out. This will give you experience talking to others to learn about a situation, proposing possible approaches, marshaling the effort of others, and delivering a result. This is managing! Managing in small ways leads to success managing in small ways, which leads to larger opportunities. The reward for good work is more work.
  2. Volunteer with charitable or vocational organizations and offer to help in ways that are more like 1 above over time.

Taking time off? Here’s what I expect as your manager.

You can file for time off in the relevant system and I’ll approve it. It is important to give your product team at least as much warning so that you can help them adapt to your absence.

The key things I expect here are:

  1. advance warning if at all possible
  2. file diligently (well before when warning is possible, right after if not)
  3. provide the same level of warning to your team
  4. negotiate well in advance with your team what to do to make sure things bowl along in your absence (you have things ready, what slack will others need to take up, what can wait for your return)
  5. ask for help if needed
  6. negotiate one week in advance with your team what to do to make sure things bowl along in your absence
  7. ask for help if needed
  8. check on how things went once you return and negotiate the new way forward from there (given that conditions may have changed)
  9. ask for help if needed

In general I think of filing for PTO as a “vacation warning” rather than a “vacation request.”

Empathy for the leader whose company is being acquired

To be a leader planning for one’s firm to be acquired is to be in conflict with oneself: one must both

  • proceed as if nothing will change, as there’s no deal until the deal is complete, and
  • plan the future in which everything will change, rendering the rod that went into “proceed as if nothing will change” entirely moot.

Along the way, especially if one of the groups (acquirer or acquired) is a publicly-traded firm, one must hide the coming deal from all but the very few people who are involved in accomplishing it. This is necessary and difficult and thankless.

Difficult because you have information that you cannot share that will have a sudden and material effect on the lives of people you’ve supported and come to care about. Difficult because the working of the deal puts an awkward gap in whatever plans you have together, and so you must dodge, delay, make excuses. Difficult because if the deal falls through you’ll have to pick up the pieces and paper over breakage caused by these gaps. Few will know why or what happened, just that you were unaccountably ineffective for a while.

Thankless because when the deal does happen all of your vagueness during the gap will be revealed as lies, lies that people won’t understand, lies that hurt to tell as you told them and hurt again as the people you deceived bark at you for deceiving them. Which you had to do.

Thankless too because any negative consequence of the deal, however indirect, is also on your head. Your reputation among those affected will be harmed. And yet it was probably the right thing to have done, to pursue the acquisition, to draw investment toward the idea that you gathered these people together to work on, or you would not have done it. Sad that the result is pain and resentment and blame.

Empathy for the employee that arrives via acquisition

An employee who comes to your team via an acquisition

  • was likely in the dark until the acquisition was announced
  • didn’t apply to be part of your company, your team, or for their new job
  • is accustomed to working on different things, with different people, toward different goals
  • doesn’t know you, or anyone on your team, or your company
  • is unaware of your expectations of them
  • experiences a loss of the status they enjoyed as a valued employee prior to the acquisition

As a result, they are likely very uncomfortable with the situation. And it’s really not at all fair; they didn’t ask for this to happen. Suddenly they are asked to adapt to a hundred dimensions of unknown.

What I learned from my big mistake

On Friday last week I deleted a chunk of the wiki. All of the product management pages, in fact. The work of a team of fifteen, organizing and guiding the work of a hundred people, gone. I’ve managed to avoid being fired, so far. People have been very nice about this, so far. But I don’t expect their good will to last: as they trip over the lack of things that were once in place they will learn the painful extent of my big mistake.

There were many failures involved in this big mistake, but the biggest failure belongs to me.

What happened

The design team, which I lead, had been putting our recent work on the design system, user research, interaction guidelines, etc. in the product management space for quite a while now, but there was a disused old space that contained similar but outdated content, penned by past designers, that cluttered up search results and created confusion among the design and product teams. I was working through the contents of this area, moving some pages to the product management space and marking them as old work, archiving other pages, and deleting “stubs” (pages devoid of content) outright. So I was working in two spaces at once.

At some point the disused old space was effectively empty, so I thought I’d delete it. From the homepage of the disused space I went to space administration, and chose to delete the space. And since I had just done a lot of work there to clear it out, and thought of that work as now safe in the product management space, I didn’t pay super-close attention to the confirmation message that warned that the deletion was not reversible. But that confirmation was attempting to warn me that I was deleting the product management space. I blithely confirmed the deletion, then a progress bar appeared, and grew, and was not cancelable.

As the progress bar grew I realized what I had done and panic rose in my chest. I quickly opened a new window and quickly did an export of the product management space, which was in the process of being deleted. I got 22MB of data, so not nothing, but that export is likely partial; I have no idea how large a full export would have been.

Expecting that I had just caused my imminent firing, I immediately Slacked someone in IT for help, my VP of Product to tell her what I had done, and the product team as a whole to apologize and point out that I would do my best to find a solution. As I said, people have been very nice about this, so far. I hope I can act in such a way that preserves at least some of that good will.

I hope they reduce my rights on the system significantly.

Lessons for me

I was working in two spaces at once, so the likelihood of “pedal error,” choosing the wrong action believing it is the right one, was high.

I’ll probably never know if it was pedal error or a bug that caused the wrong space to be offered to me for deletion. I believed I had selected the correct space for deletion. But sure of myself in that moment I skimmed over the confirmation step. Others have had this same complaint, if the support site is to be believed, that they selected one space for deletion and were offered a different one. Even so, I should have:

  • Read and heeded confirmation messages. Failing to do this was the core of my big mistake.
  • Taken a moment before doing anything momentous or destructive. A bit of time can be room for your brain to catch up to your hands.
  • Taken my own steps to make destructive acts recoverable. I could have done a full export of both spaces before deleting anything. Had I bothered to actually read the confirmation message I may have thought to do so. Had I paused I may have thought to do so. It would take only a few minutes and be free insurance against catastrophe.
  • Made use of archival or “logical delete” rather than destructive processes. When available, archiving a page or a space or other record is inherently more recoverable than deleting. It’s not clear if this was an option in my case, but it’s a good idea. If you find you never visit these archived items, you can choose to delete them later.

I’m happy that I had already learned these lessons:

  • Hang a lantern on your problem. Had I not started communicating immediately, I’m sure things would be much worse for me. Worse, had I said nothing, I would eventually have been found out and fired not for the mistake but for the cover-up. In my panic it did take a half a beat to recall this lesson.
  • Form a plan with multiple paths to success. It’s not safe to rely on a single path to disaster recovery. How many paths to success can you think of? Can you pursue all of them? How can you maximize your chance of success? If you have just one plan, and it fails, then what?

Lessons for interfaces and systems

If you would like your system to be kind to me and other normal folks, you’ll want to arrange things so mistakes are harder to make, or easier to recover from, or both. There are a lot of things you can do, roughly in order from least to most difficult:

  • Offer recovery preparation activities alongside the destructive action. It’s straightforward to add a reminder to make an export or backup to your confirmation message. Would it have helped me? Who can say.
  • Offer recovery preparation activities alongside the confirmation message. It’s a little less straightforward to offer an export at the point of confirmation. But it’s not that hard, and that reminder will catch some folks before they make a mistake.
  • Make confirmation of destructive acts require reading to complete. You’ve seen this in Figma and other places, where you are asked to type something to confirm the action. The best of these ask you to think a little bit; “Type the name of the space to confirm that you want to delete it” is better than “type DELETE to confirm that you want to delete this space” as it requires a bit more attention. In this case, a little cognitive load is your friend.
  • Make lengthy or multi-step destructive acts cancelable. Even if part of the information is saved it is better than watching the system slowly put your mistake into practice with no recourse. In my case, deleting a space involves several transactions – making each page a hidden page, removing the association of the space with its home or overview page, deleting each page, then deleting the now empty space. Stopping at any point in that chain of events would have spared some fraction of the data.
  • Offer archival or logical delete in addition to outright deletion. Users are more likely to mishandle their data if alternatives to deletion are not available.
  • Offer archival or logical delete instead of outright deletion. Users can’t delete if they can’t delete This increases data storage costs, and signs you up to write more interfaces to browse and change the status of archived records, but major mistakes become minor ones.
  • Automatically perform recovery preparation activities. An automatic export prior to deletion adds time and storage cost to the process, but can save someone’s day by giving them the tools needed to undo a mistake.
  • Make destructive acts easily recoverable. This is the most expensive yet most polite way to handle user data. Let any act be undoable. It’s not easy to architect, much less to implement, but the level of comfort it offers to users is extreme. Mistakes in word processors used to be costly and painful, but today they are trivialities. And the effort it takes away from users is likely worth the design and engineering costs.

Lessons for organizations

A prior IT regime read about the backup practices of our vendor and felt that they were sufficient protection. They read that the vendor makes nightly backups and retains them for thirty days. But there they stopped reading, not noticing that these backups are not offered to clients and are only for vendor-level disaster recovery. So while a backup technically exists, it is no protection from client-caused mishaps. There are several lessons in this one fact:

  • Do not expect your users not to make mistakes. See “lessons for me” above to learn of typical mistakes.
  • Read the whole contract.
  • Do not leave backup solely up to your vendor, even if their backup plan sounds adequate. if you aren’t in control of your data, you aren’t in control of your data.
  • Do not believe you have a working backup unless you’ve practiced recovering via that backup. What does it take to recover from a disaster? How do you know? Are you really ready to accomplish this? One way to find out.
  • Do not believe you have a working backup unless you’ve verified its contents. Ah, you have a backup. Your’e covered! Unless…drat, the backup is corrupted, or incomplete, or old because the job has been failing for a while, or a thousand other problems.

There’s also the small matter of roles and privileges:

  • The less safe the data is, the fewer people should be allowed to harm it. No backup? Restrict deletion privileges to relatively inaccessible people, not day-to-day editors. Yes, they’ll grumble. But it is for their own good.

My plan, and what I’ve done so far

I’ve got a partial export, a backup locked up at the vendor that I allegedly do not have access to, an unknown level of inconvenience caused to coworkers, and an unknown amount of time and effort between now and whatever level of recovery I am able to effect. Above I mentioned having and putting into practice multiple paths to success. The possible paths to success seem to be

  • restoring the export and finding out what is in it, in hopes that it meaningfully reduces the loss
  • appealing to the vendor to have access to their backup, which should be complete
  • scouring the existing IT infrastructure for any possible existing export or backup we might have

I am pursuing all of these in parallel. In the meantime I have

  • communicated this plan with my supervisor, with my employees, and with my peers and others on the product team
  • apologized to the design and product teams and individually to select people, especially those who had recently contributed or shared work via the wiki
  • shared my plan with the current IT regime so that we can help each other
  • opened a temporary space for the teams to use to avoid namespace pollution or any untoward effects of a possible full, partial, or botched restore

I am also communicating the emerging status with team members daily.

I know that with all of this, the likelihood of a full recovery is slim. And I know that at a certain point, some organizations will fire an employee for the effect of their mistakes regardless of the circumstances. If that comes to pass I will understand. But I will pursue multiple paths to success until that happens.

Update

The product management area of the wiki is back. My export was not usable, but the vendor agreed to help by rolling the wiki back to the moment prior to deletion, allowing us to export the data for the product management space, rolling it forward again, and allowing us to re-import the exported data. Thanks be to the IT person who convinced the vendor to help and who worked on the problem after-hours so that on the morning of day six we had the data back in place.

A side-effect of this is that every inbound link to pages in this area is broken. But so far this seems surmountable. Another side effect is that we now have a backup strategy that we are in charge of, and will not have to appeal to the vendor for further help if additional mistakes are made. I expect a security/privileges audit will come soon.

Daniel Abrahams asks: “If a task takes six hours to complete in the office, but that same employee finishes out in four working from home, who should get the saved two hours?”

If a task takes six hours to complete in the office, but that same employee finishes out in four working from home, who should get the saved two hours? The company ('back to work!') or the employee ('free time')?

I don’t like either of these options. Of course, the employee in me wants to be rewarded for producing efficiently, and the manager in me wants to take advantage of that efficiency to improve the business overall; this is the simple conflict that the question hinges on. But Daniel’s question and its two options miss the mark entirely because they assume that the employee is being paid either for their time, i.e. selling hours, or by the task or piece, i.e. selling some number of completions of a well-defined task.

This is not how I want to arrange my work or the work of the people I support. I am not paying them for their time, and I am not paying them by the piece. And I can’t really; as knowledge workers

  • our work is not so well-defined as to be sensibly thought of as piecework, and
  • our time is not a good unto itself, rather it is the way we apply that time that produces results that are valuable.

Knowledge workers negotiate and accept anticipated results they are to participate in producing, and then act largely as they see fit to do so. They are paid a salary and evaluated based on their contribution to these results. It’s hard to do a good job without some amount of continuous improvement, as we are learning as we go, and a valued contributor will find ways to do a little better than last time.

That improvement has a few rewards:

  • it produces a sense of accomplishment,
  • it creates success for the team and organization, and
  • it makes the job easier over time,
  • it frees up some effort to work on more interesting things,
  • it makes space for the employee to make decisions about balancing their ambition and their need for rest.

In the ideal case, all five of these rewards are produced.

A quick note about presenting design work in a meeting with engineering

There are a few ways these meetings might be organized depending on where we are in the design-development process.

  • early concept: help me think about the problem
  • late concept: help me choose among these ideas and develop one
  • early detail: this is where we are going, what caveats remain?
  • late detail: here’s a the detail, what questions do I need to address?

These depend on the thought that there are a few phases/horizons to product design work, namely

  1. Problem Framing (aka Research or Horizon 3), in which we learn about the challenges and needs of users to understand and select a problem to go after with concepts,
  2. Concept (or Horizon 2), in which we develop concepts and select among them with the help of users and engineering, and
  3. Detail (or Horizon 1), in which we develop the selected concept and test the fitness of our more detailed work with users as we go.

It’s a common anti-pattern that engineering gets involved only when we are in detail, but we’d like to pull them into the other phases to be informed about the available problems and to help us develop and select among concepts.

Some Agile purists will decry this phasing as anti-agile, but that requires a blinkered view of what Agile means, which often comes about when one confuses Scrum for Agile. (The dirty little secret of Scrum is that there’s a lot of hand-waving about the large quantity of work that needs to happen outside of sprints to make Scrum possible.)

Do you have some key metrics that you are measuring your team on (number of items added to the pattern library, number of customer interviews, etc)?

I tend not to be in favor of output metrics like these; they’ve been shown to be counterproductive for developers (lines of code, commits, bugs crushed, etc.), for example. But I am thinking about how best to apply the process maturity scorecard to the work of the individual designers to try to coach them on their projects. If a designer is helping to improve the process maturity of their product team(s), that’s a good sign.

On the other hand, some anti-metrics (trouble indicators) are pretty obvious – if there’s a lot of rework coming back to a designer, if deliverables are sloppy, if product managers and engineers are dissatisfied or under-involved, then we can tell there is a problem. But the opposite of these are not great metrics for individual performance, as a certain amount of rework will be technically driven, we don’t want to create an inspection-heavy situation where each deliverable is scrutinized for lack of sloppiness, etc.

I do have expectations that designers will bring work to and contribute to critique, seek my counsel on design challenges, press for collaboration, etc. But do I want to count these behaviors? Probably not.
The ideal thing would be to peg team performance to the product metrics we hope to install. The metrics we really care about are user success metrics in a product, and these are team metrics, not individual performance metrics.

So when it comes time to evaluate a designer, I like to use four Cs – customer, collaboration, culture, and content.

  1. Customer – is the work produced by the designer informed by the customer, appropriate to the customer, accepted by the customer?
    1. Informed a leading indicator, evidenced by their participation in research and testing. They are or are not doing this. Exactly how much is situational, nonnumeric.
    2. Appropriate is somewhat evident in testing, and ultimately evident in adoption, a lagging indicator (and highly dependent on product and engineering decisions, the appropriateness of the feature idea in the first place, etc.).
    3. Accepted is essentially adoption. The same caveats apply.
  2. Collaboration – is the designer an effective collaborator on their project teams? Do they work well with their peers and their project teams?
  3. Culture – what is the designer’s contribution to the culture? Are they intellectually and emotionally engaged in a way that improves their teams and the org, is neutral, or detracts from their teams and the org? Are they improving and helping the culture improve?
  4. Content – is the designer’s work organized, communicative, complete, insightful, timely? Are their skills and attention to detail usefully on display in their work?

The best articles out there about measuring designer performance are a) not very good and b) old, such as Measuring Designer Performance, or How do I create KPIs for my design team. So I’ve been content to use those four Cs, which are similar in concept to how IDEO evaluates their creative and engineering staff.