Blog / Making the Cut

File this one under “crotchety grumblings.” I started this post as a wish list for the future of CRISPR/Cas9, but came to the realization that it could apply to just about any sexy field. So instead, this is a wish list for biology in general. These points are related to the dichotomy between the business of science and the pursuit of knowledge. I’m sure some of these points have come up in previous posts, but I want to get this off my chest in one place. I understand that the business of science has very logically led us to a point where the items below are natural and common, but it doesn’t make them right. I’m also sure that most of these have been discussed in one form or another elsewhere, but I’m adding my $0.02. Consider this a reminder for students and postdocs who might be reading the blog.

At heart, all of the below come down to one thing. As scientists, we are trying to discover new things about the world. One assumption to most of our work is that there is an external, objective truth for us to observe. We see that truth through a glass darkly, and hence we attempt to put the external world in a form that can be understood from human context. We can do no else. To purposefully distort that truth does real harm and sets everyone back, but has unfortunately become de rigueur in some contexts.

1. Rickety Tricks. Doing tech dev or poking in to biology by taking advantage of overly-specific proof of concept just creates frustration and eventually casts a black cloud over whole fields. Science is typically bleeding edge, and it can be very difficult to know the boundaries of an approach while it’s being developed. But that does’t justify purposefully take advantage of “tricks” to look successful while simultaneously knowing that there’s no clear path anywhere other than the one-off proof of concept experiment. This cynical approach might inflate one’s own CV, but generates mistrust for whole fields and poisons the potential for real world application.
2. Reproducibility. Related to the above, there have been several articles about the reproducibility problem in pre-clinical science. I won’t bother to link to all of them, since by this point the problem is well-recognized and I could only accidentally miss some attributions. The general problem is that perverse incentives for publication mean that many pieces of research are just plain wrong. This isn’t necessarily outright fraud, but it can stem from squinting at the data in just the right way and hoping, because positive results are good but there’s no Journal of Negative Results. I saw much less of this when I was in industry, perhaps because the motivations are much different. If one is just aiming to get a paper out the door, then one might not particularly care if a discovery is Right or Reproducible (though one should!). But in industry, that paper (or internal discovery) is just the start for what might be a very expensive and long-running drug discovery project, and there are many incentives for robustness and reproducibility in that case.
3. Over Hyping. Be honest those who consume your science (both primary articles and press releases + interviews) and yourself. As we approach an age of open science, it’s becoming easier for non-scientists to access research articles and learn about cutting-edge developments. This is a very good thing, since non-scientists are frequently very motivated, for example by family members suffering from a disease. These people should have access to all the information they might want. But coming from outside the business of science, they might not understand the hype machine. Promising the world based on an incremental advance might increase one’s chances for getting into Nature, but has very real consequences by potentially giving people false hope. I feel very strongly here, since I’ve personally answered emails and telephone calls from people who saw certain overhyped papers and called me to ask about cures for their sick child. It’s heart-breaking. And if compassion for fellow humans doesn’t motivate, consider that the government (including people who set budgets!) have little tolerance for science that promises then doesn’t deliver.

It’s a beautifully sunny day on the weekend, so I’ll stop here before I dip into too much of a funk. But I’ll end by saying that these are things we can fix. There is no Science with a capital S. We are scientists who are building science as we go along! We are a global community – we write our own papers, referee our own papers, write our own blog posts, form our own societies, and organize our own conferences. It’s all up to us.

X Close

I started writing a lengthy analysis about whether or not Cas9 is specific. It contained several in-depth analyses of many papers. There were arguments for and against. But right in the middle I realized that the details of the literature surrounding specificity don’t really matter and I deleted the whole thing. Here’s why…

Cas9’s specificity might be pretty interesting to you if you’re creating cell lines for research use. After all, you don’t want to be reporting a phenotype that actually stems from some off-target knockout. But if you’re thinking about a gene correction therapy, specificity will keep you up at night in a cold sweat.

And that distinction, together with the mindset of research vs therapeutics, is the key. Several papers have shown that Cas9 is both moderately specific and moderately permissive. There’s all kinds of literature about seed regions and sgRNA bubbles, and so on. But right now, we can say that sgRNAs can be at least pretty good, and it’s hard to tell when they’ll be bad.

So for research purposes, it doesn’t matter whether or not Cas9 is highly specific, because you should just choose two distinct guides and demonstrate that your phenotype is robust to choice of guides. The chance that two guides of different sequence will have the same off-target is very low. This is much like what’s done with genome-wide CRISPRcut/i/a libraries, but I think it should also extend to CRISPR-made cell lines and probably even animal models.

And for therapeutic purposes, it doesn’t really matter what the literature says. Even if all papers everywhere said that Cas9 was absolutely stringent, you’d still need to demonstrate specificity (or at least knowledge of benign off-targets) for your application of interest. Anyone who would use a genome-targeting reagent in humans without careful homework on that reagent (regardless of literature precedent) has no business making therapeutics.

X Close

The field of genome editing is moving at breakneck speed and protocols are rapidly evolving. We’ve already made a few different posts on tips, tricks, and protocols for genome editing and regulation. But effectively sharing protocols and making sure that they’re up to date is a daunting task. Much better to have a community-driven effort, where a starter protocol can be tweaked and updated as new developments come along.

That’s why I’m happy to share that we’ve recently started putting our methods on Protocols.io. This is an open repository for protocols, which the great feature of “forking”. This means you can start from a protocol that you like, tweak it as desired, make a record of the tweaks, and re-publish your changes. Everything is also linkable to a DOI, which means you can potentially reference online protocols from within papers.

IGI protocols for T7E1 assays, in vitro transcription of guide RNAs, Cas9 RNP nucleofection, and more are available at https://www.protocols.io/g/innovative-genomics-initiative/protocols. 

Here’s an explanatory video, from the protocols.io team.

X Close

For the last couple of months, a few of us at the IGI have been generating new sgRNA libraries for CRISPRi and CRISPRa. After scraping colonies off of nearly one hundred extra-large LB-Agar plates, it was time to fill the lab with the sweet smell of lysed bacteria and DNA prep buffers. We were working with 21 separate sublibraries, totaling around 250,000 sgRNAs. Plasmid prep on this scale is a completely different beast from anything I had done before, so we decided to share some thoughts on what works (and what doesn’t!) for efficiently prepping sgRNA libraries.

Prepping the work station

We were worried about other plasmids sneaking into our preps–especially individual sgRNA plasmids that get used frequently in our lab. We doused and scrubbed our benches and vacuum manifold with 70 % ethanol and RNase-Away before starting, and a few times throughout the day. This should hopefully destroy or denature any stray plasmids hanging around. It’s also worth cleaning out your vacuum trap and putting fresh filters in the vacuum line, since old dirty filters can really weaken vacuum power.

Do all the DNA prep at once

For me, it’s much more efficient to spend a couple of days solely devoted to high-throughput DNA prep than to spread the work out over several days, a few columns at a time. 

Teamwork

The initial lysis and neutralization steps in most plasmid preps are time-sensitive, so there’s a limit on how many samples one person can process at once. We found that a team of 3 people (each processing 8 samples at once) maximized our throughput without us bumping into each other too much. After eluting DNA off the columns, once person can manage the DNA precipitation while others start on the next round of samples.

Starting material

Scraping the colonies off of a 23×23 cm LB-Agar plate gave us an average bacterial pellet mass of 1.1 g (range 0.5-1.6 g). This meant that each plate of bugs got its own maxiprep column (see below for kit recommendations). If you’re working with bugs from liquid culture or other plate sizes, you can pool or aliquot the samples to get a similar pellet mass per column.

Plasmid prep kits

We wound up trying several different plasmid prep kits, and the clear winner in our hands was the Sigma GenElute HP Plasmid Maxiprep Kit. The columns are compatible with the QIAGEN 24-port vacuum manifold we already had in the lab, the protocol was amenable to doing 24 preps in a batch, and the house vacuum system in our building was strong enough to pull liquid through all 24 columns at once. Importantly, all of the columns ran consistently and reasonably quickly. One slow or plugged column is an annoying but solvable problem when doing 4 or 5 preps, but it can really back up the pipeline when doing multiple batches of 24. Our  average yield from this kit was 1.4 mg per prep.

Kits to avoid:

• Sigma GenElute HP Plasmid Megaprep: Sigma advertises 4 times the yield from a megaprep column compared to their maxipreps. Some of our samples could be pooled, so we thought pooling 4 samples into one megaprep would be faster than running them as 4 individual maxipreps. Boy were we wrong! The megapreps had to be processed one or two at a time, and thus didn’t scale well at all. Worst of all, the megaprep columns were NOT compatible with the QIAGEN vacuum manifold. We managed to fix this with tubing and adapters, but the house vacuum system was only strong enough to pull on one or two of the larger megaprep columns at a time. For us, mega preps took far more time and gave about half the yield we would have expected from just grinding through 4 times as many maxipreps.
• QIAGEN Plasmid Plus Maxiprep Kit: 1 out of the 8 columns we used  stalled while running the cleared lysate. That column had to be left on the vacuum overnight. Our yields were also lower than the Sigma maxipreps. 
• QIAGEN HiSpeed Plasmid Maxiprep Kit: These don’t scale well at all. The columns aren’t compatible with a vacuum manifold, and the QIAprecipitator syringe filters require a lot of manipulations to each individual sample. After the first 4 samples, I ditched the QIAprecipitator step altogether. Precipitating the DNA with a 45 minute spin was much faster when dealing with 10 or 20 preps at once.

We’re always interested in ways to make the next sgRNA library prep easier than the last. If you have your own favorite plasmid prep kit or other tricks for efficient library preparation, feel free to leave a comment. 

Special thanks to the other members of Team DNA Prep–Gemma Curie, Amos Liang, and Emily Lingeman. I’d still be running maxipreps if it weren’t for them!

X Close

Following up on my previous post about genome-wide CRISPR libraries, I thought it would be useful to show a bit more.

There are many things to consider when doing library work, but two major ones are 

1. How sure are you that a hit stems from on-target activity vs off-target trickery?
2. What fraction of the library is functional?

On-vs-off target is the real worry, since you could spend a great deal of time chasing down spurious hits. CRISPR (and sh/siRNA) libraries tackle this problem with redundancy, and one should always require that a phenotype enrich multiple guides corresponding to the same gene. But in libraries with relatively low redundancy (e.g. GeCKOv1 only has 3-4 guides per gene), it’s easy to become enamored by a hit with a red-hot phenotype but only one guide. 

The concern about functional fraction of the library is more technical, but impacts both ease of the screen and the redundancy point from above. If many of your guides are non-functional, all that extra work to clone and transduce your massive library vs a smaller one is wasted effort. Worse, your chance at redundancy is diminished with each non-functional guide.

With that in mind, here are updated distributions for existing genome-wide guide libraries targeting human cells. The “penalty” axis is in log scale, and the penalties are easily interpretable to highlight the class of the problem. For penalties, the tens place represents the score of a guide itself, 100s place represents number of intergenic off-targets, and 1000s place represents genic offtargets.

For example, Anything with log(penalty)=0-2 has no off-targets and could be OK, though guides have much higher chance to be completely non-functional as one approaches 2. log(penalty)=2-3 have intergenic off-targets, with each 100-spike an additional off-target hit (e.g. penalty of 100 = one off-target, 200 = two off-targets, etc). log(penalty)=3-4 contain Pol III terminator sequences and are probably never even transcribed. log(penalty)=4+ have genic off-targets, with each 100-spike an additional off-target that impinges on a gene (e.g. penalty of 1000 = one off-target, 2000 = two off-targets). These penalties compound, so a score of 2,354 means two genic off-targets, 3 intergenic off-targets, and a guide penalty of 54.

Note that calling genic vs intergenic is done using Ensembl data and is sensitive to the type of CRISPR experiment. CRISPRi looks for hits within -50 to +300 of a gene, while CRISPRcutting looks at exons (for the moment we’ll leave aside the scary prospect of cutting within potentially functional intronic or UTR regions).

In general, things are looking pretty good for CRISPRi. There’s a bit of an advantage here, since CRISPRi only seems to work in a narrow window around the transcription start site, and so off-targets are less likely to hit a gene. CRISPRcutting libraries are not doing all that well with off-targets in annotated exons, and only deeper per-guide analysis would tell whether guide redundancy takes care of mis-called phenotypes. It’s nice to see that GeCKO has improved with v2 (e.g. got rid of terminator sequences), and hopefully v3 can get some of the genic off-targets under control.

I want to stress that all of these libraries work just fine and have been used successfully to give biological insight. But keep these guide properties in mind when working with each library and thinking about hits arising from their use.

X Close