Профиль

congrats!

cyto is free, open-source, and production-ready. Built in Rust for reliability at scale. Currently supports 10x Flex GEX and CRISPR, with more modalities coming. Try it out and let us know how it works for you! github.com/ArcInstitute...

GitHub - ArcInstitute/cyto: A mapper for single cell sequencing reads with abstract geometriesA mapper for single cell sequencing reads with abstract geometries - ArcInstitute/cytogithub.com

cyto is the first large-scale bioinformatics project to build with BINSEQ. Switching to BINSEQ can achieve mapping rates of 50M reads per second reduce your storage requirements by about 40%. github.com/ArcInstitute...

GitHub - ArcInstitute/binseq: A high efficiency binary format for sequencing dataA high efficiency binary format for sequencing data - ArcInstitute/binseqgithub.com

We also show that we can reproduce the results of CellRanger at a fraction of the resource cost. Our concordance is above 99.85% as measured via Spearman on matched cell UMIs and our lower-dimensional representations show perfect overlap with no method specific clustering.

cyto was built from the ground up to be modular and to expose the individual modules to the user. Each step is highly optimized and can be run independently, perfect for production scale workflows as it allows for better parallelization and resource allocation on smaller nodes

Currently the only tool that supports this data type is CellRanger and we show that cyto provides runtimes an order of magnitude faster (16x), uses less than half the memory, dramatically reduces CPU-hours (30x) and reduces total IO by more than 5x.

I've tried this at least 3 times haha I think honestly the best way to do it is not really to port it but drastically rework the way that its written.

Maybe this will be the year we start to really question the foundational infrastructure of the field. Obligatory BINSEQ mention here - keep a lookout the next couple weeks for an update! github.com/arcInstitute...

GitHub - ArcInstitute/binseq: A high efficiency binary format for sequencing dataA high efficiency binary format for sequencing data - ArcInstitute/binseqgithub.com

My attempt at the 15th competing standard: github.com/arcInstitute...

GitHub - ArcInstitute/binseq: A high efficiency binary format for sequencing dataA high efficiency binary format for sequencing data - ArcInstitute/binseqgithub.com

To be one with the borrow checker one must first be willing to let go

Check out the github and give it a shot! github.com/arcinstitute...

We can make as many pipes as we have threads, each with a fixed record range and with a specified segment (R1 / R2). Then we can connect to each pipe on a reader and treat it as a normal FASTX file. What's great about this is we can process *either* sequentially or in parallel *without* deadlocks!

This was a fun engineering problem but ultimately was not very difficult because of the way BINSEQ is designed in the first place! Named pipes can be a headache because it requires coordination between readers and writers but because BINSEQ is random access the implementation is straightforward.

ARM64 linux I think is pretty common on cloud computing environments. Might be worth to build for it also

Built with uv so you don't have to worry about the dependencies or environments. Simple as: ``` uv tool install anntools-bio anntools --help ```

Side note: sandbox.bio is so cool. Setting up an environment where you can learn and play around with these tools in the browser is no simple feat and I think it's an excellent educational resource for the bioinformatics community. I'm very happy and proud to contribute to it!

The pattern counting is something I'm especially stoked about. I was actually very surprised to see that this feature isn't more common on grep-like tools (outside of bioinformatics as well). I've had this problem for years and I end up writing bespoke tools that do some variation of it.

And stay on the look out the next couple weeks (hopefully) for the release of an even bigger project built with binseq!

And if you're interested in building with binseq here is the place to start! github.com/arcinstitute...

GitHub - ArcInstitute/binseq: A high efficiency binary format for sequencing dataA high efficiency binary format for sequencing data - ArcInstitute/binseqgithub.com

I've also added some nice functionality to bqtools including a very useful colored grep! github.com/arcinstitute...

GitHub - ArcInstitute/bqtools: A command line utilty for working with BINSEQ filesA command line utilty for working with BINSEQ files - ArcInstitute/bqtoolsgithub.com