this post was submitted on 01 Aug 2024
1 points (100.0% liked)

Free Open-Source Artificial Intelligence

2889 readers
1 users here now

Welcome to Free Open-Source Artificial Intelligence!

We are a community dedicated to forwarding the availability and access to:

Free Open Source Artificial Intelligence (F.O.S.A.I.)

More AI Communities

LLM Leaderboards

Developer Resources

GitHub Projects

FOSAI Time Capsule

founded 1 year ago
MODERATORS
[email protected]
[email protected]
1
Local, privacy respecting, open source LLM that can transcribe phone calls and summarize the call? (lemmy.ml)
submitted 3 months ago by [email protected] to c/[email protected]
5 comments fedilink hide all child comments
 

Hi everybody, I find a huge part of my job is talking to colleagues and clients and at the end of those phone calls, I have to write a summary of what happened, plus any key points that I need to focus on followup.

I figured it would be an excellent task for a LLM.

It would need intercept the phone call dialogue, and transcribe the dialogue.

Then afterwards I would want to summarize it.

I'm not talking about teams meetings or anything like that, I'm talking a traditional phone call, via a mobile phone to another phone.

I understand that that could be two different pieces of software, and that would be fine, but I am wondering if there is any such tool out there, or a tool in the making?

If you have any leads, I'd love to hear them.

Thank you so much

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 0 points 3 months ago (2 children)

Haven't heard of all-in-one solutions, but once you have a recording, whisper.cpp can do the transcription:

The underlying Whisper models are MIT.

Then you can use any LLM inference engine, e.g. llama.cpp, and ask the model of your choice to summarise the transcript:

You can also write a small bash/python script to make the process a bit more automatic.

[–] [email protected] 0 points 3 months ago (2 children)

Okay, the idea is excellent, but I've just spent the last hour trying to get any app out there to record my calls.

I've tried the open source one on f droid, and it almost works. I can get it to record my side, but that's it.

I tried commercial ones. I tried commercial ones with horrendous privacy policies. Nothing seems to work.

I've used the accessibility options. I've gone deep down into the rabbit hole, so it looks like Android is fully cutting off the ability to record calls. In Australia at least.

What a shame.

These apps all seem to have the same ability of dropping the recording into a folder, so I could synchronize that across my network, have my server check for new files that appear into that folder, and then the LLM could convert that into a text file and send it straight back to me.

Living the dream! But... Not

[–] [email protected] 0 points 3 months ago* (last edited 3 months ago)

Basically depends on if it's legal to unilaterally record in your jurisdiction, if not the apps won't be made available on pain of lawsuit. Nothing stopping you using speakerphone and recording with something else tho. Again, depending on jurisdiction, you may need to CYA with 'your calls may be recorded for training purpose'. Note that unilaterally recording (i.e. without notifying the other party) is often a felony.

[–] [email protected] 0 points 3 months ago* (last edited 3 months ago)

I expected that recording would be the hard part.

I think some of the open-source ones should work if your phone is rooted?

I've heard that Google's phone app can record calls (though it says it aloud when starting the recording). Of course, it wouldn't work if Google thinks it shouldn't in your region.

By the way, Bluetooth headphones can have both speakers and a microphone. And Android can't tell a peripheral device what it should or shouldn't do with audio streams. Sounds like a fun DIY project if you're into it, or maybe somebody sells these already.

[–] [email protected] 0 points 3 months ago

That isn't such a bad idea.

I could probably semi automate a lot of this stuff as long as I can get something to record the phone call.

I'll go down that route and see what I can find. Thank you so much I do appreciate it