Something that works out nicely in uses of streaming is putting chunked results in the functor layer. We can do this with e.g. Of (Vector a) but consumers of this may be tempted to use sequence to get at the results, but (without magic fusion rules) this will necessarily create garbage through short-lived Step constructors for every element of the inner vector/chunk.
What streaming could do instead is generalise functions like Streaming.Prelude.mapM_ to support arbitrary Bifoldables instead of just Of, like:
mapM_ :: Monad m => (a -> m x) -> Stream (Of a) m r -> m r
mapM_ f = loop where
loop str = case str of
Return r -> return r
Effect m -> m >>= loop
Step (a :> as) -> f a *> loop as
Can be instead:
mapM_ :: (Monad m, Bifoldable of) => (a -> m x) -> Stream (of a) m r -> m r
mapM_ f = loop where
loop str = case str of
Return r -> return r
Effect m -> m >>= loop
Step step -> bifor_ step f loop
Which would allow the of functor to be anything like data Of a b = !a :< b or data OfChunk a b = OfChunk !(Array a) b and still be able to stream out single results at a time without any extra operations. I think the basically same generalisation can be applied to a lot of the consuming functions like toList, but I haven't looked closely.
Downsides: More compiclated, uglier type signatures.
Alternatives: Just use S.mapM_ (Vector.mapM_ f) and similar.
Something that works out nicely in uses of
streamingis putting chunked results in the functor layer. We can do this with e.g.Of (Vector a)but consumers of this may be tempted to usesequenceto get at the results, but (without magic fusion rules) this will necessarily create garbage through short-livedStepconstructors for every element of the inner vector/chunk.What streaming could do instead is generalise functions like
Streaming.Prelude.mapM_to support arbitraryBifoldables instead of justOf, like:Can be instead:
Which would allow the
offunctor to be anything likedata Of a b = !a :< bordata OfChunk a b = OfChunk !(Array a) band still be able to stream out single results at a time without any extra operations. I think the basically same generalisation can be applied to a lot of the consuming functions liketoList, but I haven't looked closely.Downsides: More compiclated, uglier type signatures.
Alternatives: Just use
S.mapM_ (Vector.mapM_ f)and similar.