2025: A Year of Progress

By Tyler Logtenberg

2025 has been a huge year for Datastreamer's customers. Here are all the major releases that are helping our customers' pipelines bring new capabilities to market, faster.

Many new data sources fed into customer pipelines

For customers, the ability to rapidly integrate new data sources has enabled them to free up major engineering time.

Some of the highlights were:

• Reddit (multiple providers)
• Apify marketplace (with over 5k+ options)
• YouTube (multiple providers)
• Snapchat
• Pinterest
• Google Search
• Google Maps
• TikTok
• Twitter+ Firehoses
• Auto Sources (more on that next!)

Auto Sources freed customers from selection stress

Many customers gave us the feedback and insights that they just needed stable data. The choosing of providers, merging them, comparing them, was too much.

Auto Sources is the answer to that.

Use Auto Sources within your pipelines, and it handles the right provider selection for that query, and conversion to a common format and schema. Allowing you to just ingest, without all the stress.

Datastreamer rolled out an in-pipeline storage option

Pipelines are great, but pulling and pushing data from pipelines can be difficult.

Searchable Storage is Datastreamer's own offering to use high-performance searchable storage within your pipelines.

Some customers use it for primary storage, others as a cache, and others as a manner to augment data.

It's not just storage though. It added APIs to do trend analysis, counting, histogram aggregation and more.

Storage integrations with partners too

Searchable Storage wasn't the only solution to ease storage integrations. We expanded our ecosystem with integrations to:

• Kafka
• Cloudflare R2
• Fivetran ETL
• Databricks

And options of delivery beyond specific providers:

• Webhook
• Firehose delivery
• SFTP

Storage wasn't just pipeline posts, files got a big addition.

Files, profiles, videos, and enrichments. So much possible within

Getting content into your platforms and products using pipelines running on Datastreamer is easy. Getting that data to its maximum value is the role of enrichments.

Enrichments included:

• Video analysis, transcription, logo detection, and more (from partners like SocialVoice)
• Deep content analysis of abusive content or dangerous content (from partners like Tisane)
• File and profile downloading and enrichments (from partners like WebSightLine)
• And even our own enrichments from the team at Datastreamer. Including hits like the influence and reach prediction models.

Query creation got agentic too

Queries are often tricky, understanding exactly how to get useful data with as little noise as possible.

Throughout the year, the team released:

• Query builder wizards to help with more complex sources.
• Query validation logic to help identify when a query may fail.
• And agentic query creation, using the power of LLMs to understand the objectives and design a query to get the data.

"Datastreamer is the interface layer between autonomous agents and the dynamic, high-signal world of social data." Tyler Logtenberg, CPO

Your code and your pipelines got even closer

Saving one of the best for last, Custom Functions brought the ability to add your own Python functionality right into your pipelines.

Beyond using them for integration of your own capabilities and scripts, these functions come with a boatload of templates.

These "Recipes" include functionality like:

• Bot detection
• High-level sentiment analysis
• Emoji detection
• Urgency detection
• Subjectivity scoring
• And more

Better cost insights and reports to support all the new functionality

With new features, came the need for new releases on cost control and cost prediction. To help with that, the Datastreamer team rolled out:

• Brand new billing dashboards with greater source-specific details.
• Add-On estimator to forecast and estimate future spend.
• Budget and usage alerting to measure and alert on cost milestones.
• In-platform support elements, like tagging of content, deduplication components, and other filtering capabilities. Removing noise inside the flow of data, and reducing downstream costs.

We aren't done yet. With the biggest update (customized detailing cost measurements) coming right at the start of 2026.

It wasn't just cost insights that rolled out. Detailed insights into the pipelines itself got multiple releases:

• Detailed logging was added to every component, along with detailed metrics.
• Many tools (and APIs) were rolled out to help engineers monitor, assess, and action parts of the pipelines.
• Improvements to how components recovered, handled schema changes, rate limits, updates, and more brought even greater stability to customer pipelines.

A great 2025, a greater 2026

With close to 100+ feature releases during the year, our customers helped us create the best pipeline platform for working with social and web data.

2026's releases are already in progress. The team is working to roll out updates in Auto Sources, detailed billing insights, cost forecasting, and more sources. 2026's roadmap brings even more value.

From our team at Datastreamer, we wish you a happy holidays!