At first glance, ‘Understanding the directories’ seems a rather innocuous title for a roundtable, but it is a phrase loaded with expectation. Lawyers often complain that the research and ranking process is shrouded in mystery. Despite this perceived opacity, they still want results. Which means that those of us tasked with managing the process are ultimately responsible for much of the heavy lifting – from coordinating participation and justifying relevance to striking the right balance between objective and subjective evaluation of significant case work. Yet that is only part of the battle. As firms face a shifting industry, where the differentiators of yesterday are becoming the norms of the present, independent qualitative research is becoming increasingly valued by prospective clients. In dealing with the directories, we are faced with the potential consequences of a number of unstable elements outside of our control. These elements factor heavily in the research process, although we are not always aware to what extent. This uncertainty, along with the need to have a candid dialogue around these issues, served as the initial catalyst for the conference session that this roundtable stems from. 

On 5 October 2016, I moderated a panel session at the inaugural Law Firm Marketing Summit and was joined by editors from two of the industry’s leading legal directories, Laura Mills of Chambers and Partners and Sam Duke of IFLR1000. The purpose of the session was to examine directories in depth – from implementing best practice to deciphering the credibility of the research and the rankings. Unfortunately, we ran out of time and several probing questions were left unanswered. This roundtable aims to pick up where that session left off. While there is no way we could make this roundtable all-encompassing in just one sitting, we do hope to demystify some of the key issues frequently raised by firms.

Panel Discussion

Reign Lee: This is a question I am regularly asked by lawyers who have been participating in this process a long time – what distinguishes the main legal directories from each other? With so many out there, should firms be picky about where they submit or should they be submitting to everything in order to leverage their rankings? 

Laura Mills: I think our chief distinguishing characteristic is our emphasis on carrying out in-depth telephone interviews with clients in order to gain a comprehensive picture of both their needs and preferences, and the state of the market as a whole. I also think that our research team is really unique in its size, which allows us to take plenty of time to research all of the countries we cover, and in the quality of the individual researchers, who are all extraordinarily bright and talented.

Sam Duke:The only guides that I ever hear about in a financial and corporate context are Chambers, Legal 500 and IFLR1000, this is not counting national legal magazines that may run awards or broader awards like FT Innovative Lawyers. On that basis I would say limit the number of guides you submit to. It is better to be fully engaged with a few publications rather than putting a lot of time and resources into submitting to everything.

RL: Do the directories still matter? And, how will they remain relevant as the industry shifts? 

LM: I think that, because clients speak with us, know we take their feedback seriously, and see their own thoughts and opinions reflected in the rankings and editorial, that they know we are a reliable outside source of affirmation of quality for law firms. Additionally, as law goes more international, the breadth of our guides becomes even more important. We are dedicated to covering as many jurisdictions as possible, and that's of real value for clients all over the world. 

SD: There’s always room for independent guides, particular in emerging markets where a firm’s competence is trickier to judge. However, guides need to move with the times and for a transactional focused guide such as ourselves that is what we intend to do by launching new tools and products. We want to give our audience a range of options for judging firm ability: they can look at our rankings, they can read our reviews and the client feedback, but they can also interrogate a firm’s track record and evaluate them that way.

RL: So how credible is the research and the ranking process? You are dealing with a number of unstable factors: peer feedback, independents vs. globals, the necessity of client feedback and the varying levels of experience of researchers – some of whom may not have a full understanding of the market or even the practice area.

LM: As we employ dozens of savvy, smart researchers who look at their markets as a whole and take a wide range of factors into consideration, I think our rankings are very highly credible. On the subject of peer feedback, the first thing I'd say is that, in my experience, people are very willing to give credit where it's due, so it's not as though we're wading through a litany of negative commentary. That being said, if we do get some, we would first cross-reference it with any client feedback. If a peer says they think you over-staff, but a client cites your team's lean staffing as one of its greatest strengths, there is no question as to whose opinion we would value more. We can also tell when someone has an axe to grind, so if we're getting nothing but negatives out of a source, we will disregard it. If we get a preponderance of credible negative feedback, that is something we would investigate more fully, but what people don't realize is that someone's peer feedback is far more likely to save them than to damn them.

As far as independent vs global firms go, since we judge each firm by country (or by state), it is the work being done on the ground in that country that is the most important. A lawyer in Bratislava isn't getting ranked because her colleagues in London are good, she's getting ranked because the work she does in Bratislava is excellent, and she is afforded the same opportunity as any of her peers there. While many international firms' local offices benefit from intra-firm referrals, the independents also benefit from being able to take instructions from a wide range of other firms. It is up to each firm to highlight its own strengths in its particular area, and being an independent firm doesn’t need to be an impediment to rankings. If you're doing great work and clients think you're doing it well, that is what matters most.

For us, client feedback is the cornerstone of our rankings and our greatest USP. There are plenty of great resources out there that rank firms by deal size, volume, and value, but we want to know not just that you've done the work, but that you've done it well. We want to know why you got that mandate in the first place and whether the client is likely to use you again. We're also keenly aware that clients are giving generously of their time when we speak with them, so we do our utmost to make the most of the time we have with them, and to minimize our intrusion in their work days.

While we do have dedicated training for our new researchers, and carry out internal seminars that delve into particular practice areas, much of the subject matter mastery takes place during the research phase. When the researcher is confronted with a practice they're unfamiliar with, they will undertake independent research to familiarize themselves with it. We have naturally bright and inquisitive researchers and a very open environment in which they're encouraged to ask as many questions as they need. Our editors and deputy editors have more specialized knowledge that we have all developed over our years of experience at Chambers and we oversee all of the research to ensure that each person has the appropriate understanding necessary to rank their section. 

SD: Our primary focus is transactional evidence, so a firm has to demonstrate a clear transactional track record for us to seriously consider changing their ranking. Client feedback is used to back up decisions based on the transactions and can be a factor that pushes a firm into a higher category. Peer feedback is very much a supporting factor –  it is useful to get views from the market but we wouldn’t base a ranking decision solely on this. In regards to researcher knowledge, we have internal training for all new starters to get them up to speed on the various areas of financial and corporate law, this focus also means that the researchers aren’t stretched too thin, they know that they can focus on banking and finance, capital markets, M&A and restructuring and insolvency as the core areas. Our new emphasis on deal tracking (through our soon to launch Deal Data tool) also means that our researchers are abreast of major transactions in the markets as they happen and thus are aware of trends and developments. We also work closely with our sister publication the IFLR magazine and share data and knowledge with their journalists.

RL: Is there a better way of sourcing client feedback? Lawyers have to be very careful about referee burn out, yet directories seem to want more and more names. Isn’t it better if you simply contact people independently to corroborate work highlights? 

LD: We could do that, but if we did our job well, it wouldn't result in fewer clients being contacted, and it would result in the research process taking several times longer. We would have to, for example, call the GC of a company who had a big public deal, ask them about the big picture work, but then ask to be referred to the in-house person who worked with outside counsel on the competition aspects, the in-house person who worked with outside counsel on comp, the in-house person who worked with outside counsel on tax, etc… The firms are best placed to know who the most authoritative voice is and having the contacts provided makes it a lot easier for both us and the client. Ultimately, it is incumbent upon each partner to sense their client's level of interest, which is why the first piece of advice I give to firms is "make sure the client is willing to speak with us".

SD: We have no limit on the number of referees people can submit, but we don’t punish people for not sending us referees and as mentioned above, for us, transactional evidence is the real key factor. We also give referees the option of either speaking with us via a telephone interview or completing an online form, so if referees are not comfortable speaking with journalists or don’t have time, they can still give us feedback. We are also careful not to contact clients more than once so we don’t become a bother. Ultimately, it is up to the firms as to who and how many clients they send. We try not to dictate too much.

RL: What do you have to say to those firms or those lawyers that say, ‘our work speaks for itself’?

LD:  I would say that, while we do put a great deal of emphasis on the work already, we have no idea whether: the client thought you did an amazing job on that work and can't wait to use you again; or you did a decent job and they'd be happy to go back to you if you were willing to offer them a good price; or they'll never use you again.  I would also say that it's easier for a public M&A lawyer to say that than an Anticorruption lawyer, whose work is often non-disclosable.

SD: Fine, as long as we know what that work is. For us, as mentioned, transactional evidence is key, so as long as firms let us know what they are doing we are happy to judge them on their track record.

RL: Do firms/lawyers ever get blacklisted for complaining and/or having negative interactions with a directory?

LD: This question genuinely made me laugh out loud. Absolutely not. If we didn't rank firms who complained about the rankings, we'd lose about 75% of the firms in there! But, seriously, no – we understand that the rankings can inspire a range of emotions in people and we do take critical feedback seriously (and we don't take it personally). Also, we do not drop firms or individuals who contact us and say things like "if you're going to rank me in Band 4, I don't want to be in the guide at all".

SD: No, people have every right to complain and question our decisions. We also learn from feedback so we can’t take umbrage if people provide it. What we hope to move towards with our Deal Data product is more transparency on our decision making, firms will be able to compare their work with others in the market and that should make it clear why we have made the decisions we have. 

RL: What differentiates a Band 2 firm from a Band 1 firm? 

LD: There is no one answer to this question, I'm afraid! Sometimes it's the work, sometimes it's the clients for whom you're working, sometimes it's the volume of feedback you receive, sometimes it's the quality of the feedback. It's usually a combination of those four things, though.

SD: Tier 1 firms for us are firms who work on the biggest and most complex deals. They don’t have to be the biggest, but often there is a correlation. Tier 2 firms are still very good – everyone in our rankings is, as we don’t rank every firm in the market – but Tier 1 are real stand out performers.

RL: And now the million-dollar question: how do firms go about improving their rankings? 

LM:  Making sure their clients are: (a) willing to be put forward as a reference; (b) going to reply when we reach out to them; and (c) going to say positive, detailed things about as many members of your team as possible.

SD: The best way is for firms to make us aware of all relevant deals and also to stress, whether it be in a submission or press release, why a deal is important or relevant. If the researchers do their job right, the deals should speak for themselves.

The inaugural Law Firm Marketing Summit was held on 5 October 2016 in London. The participants on this panel were:  

Laura Mills, Editor of the Chambers USA Guide. She has been editor of the USA guide for over five years, having started at Chambers & Partners as a researcher in 2008. She is responsible for all of the research-based content in the USA guide and runs a team of six deputy editors and 40-60 researchers, who produce the rankings and editorial for which Chambers is known. Her previous experience as a deputy editor included overseeing research in the Asian and Middle Eastern markets. 

Sam Duke,  Editor of IFLR1000 – The Guide To The World’s Leading Financial Law Firms. The IFLR1000 is a legal guide focusing solely on the financial and corporate markets. Sam manages the editorial team across offices in London, Hong Kong and New York. 

Reign Lee, Director, Legal Media Monitor. LMM is an international agency that specializes in media planning and press relations services exclusively for law firms. She provides strategic advice to clients on a wide range of issues, including brand development, market positioning, legal directory process management and content development. She has a wealth of experience working with independent and global law firms – as a BD and marketing adviser, communications specialist, and as a training director.