On Android, WhatsApp supports writing a database backup to the shared storage. The backup contains all messages in an sqlite file "encrypted" by an AES key which is hardcoded into the application...

This is of course ridiculous from a security point of view (not unlike using the same RC4 keystream in both directions), but including that key in TextSecure might cause legal problems for this project.

I agree, that could cause problems. Too bad they encrypt it since the encryption is useless. WhatsApp supports sending a chat via mail. It would be some extra steps, but maybe import that mail somehow?

I don't think we can do this.

I thik we can do slightly different approach.
What's app allows exporting chat in txt from https://faq.whatsapp.com/android/chats/how-to-save-your-chat-history/?lang=en

It looks like this

dd.mm.yyyy, HH:MM - Author: text

e.g.:

21.12.2019, 14:00 - John Doe: OK 😃

Using this approach we should be able to easily migrate chats.

Doing it this way would lead to a loss of the included media, but the feature would help a lot of people to get rid of whatsapp and in the end it is better than nothing.

A lot has changed since 2014: Backups are now encrypted with a per-account key that's supplied by WhatsApp servers after recovering an account as far as I know

On the other hand, there is now GDPR and other legal provisions for the right to export your own data in some form in many countries. I don't know how applicable this is to client-side data, though.

Maybe it helps that at least in Germany, chat to HTML export is no longer available (for patent reasons), so potentially WhatsApp could be required to offer another, maybe more machine-readable export format?

At least for me and I am living in Austria, WhatsApp has a export feature which gives you a plain .txt file which is formatted like a log file.

The format is like:
[date] User: message

Ok, pretty weird. At least here in Austria (and I suppose in Switzerland too?) this feature is still available. Anyway, as you already state (according to the GDPR) somehow WhatsApp must supplier you with the data they store about you and this is clearly the case with your whole chat history.

Edit: The article also states that this is only the case in Germany due to "national regulations"

I created community feature request

I would strongly upvote for this feature. I used whatsapp exclusively for everything and would like to switch to signal. But wanna shift all my msgs and groups to the new platform too. Its not just me, everyone who is planning to migrate to signal will be pleased to have this. If you enable this, you can assure that signal would surpass WhatsApp in usage.

That would be amazing. It's one of the reasons which prevents me to switch completely to Signal.

somehow WhatsApp must supplier you with the data they store about you and this is clearly the case with your whole chat history.

Does WhatApp permanently keep any message data on their servers, or are messages removed once downloaded by the client(s) (remember that chat backups are performed via Google Drive/iCloud, not through some WhatApp-operated service)? If so, I'm not sure whether GDPR applies in the usual fashion.

(I'm on the same page with all of you: There should be a way to export full chat data.)

@doersino Quoting their privacy policy:

If a message cannot be delivered immediately (for example, if you are offline), we may keep it on our servers for up to 30 days as we try to deliver it. If a message is still undelivered after 30 days, we delete it.

At least on iOS it is possible to extract WhatsApp Chats with media. There is a Software called iMazing and it can extract WhatsApp Chats to CSV, Excel, TXT, PDF including:

• Group events (who joined and who left a group and when)
• Message status: sent, delivered, read, deleted
• Contact, location and link sharing
• All types of attachments: photos, videos, audio and files
  Source: https://imazing.com/print-and-export-iphone-whatsapp-chats

somehow WhatsApp must supplier you with the data they store about you and this is clearly the case with your whole chat history.

Does WhatApp permanently keep any message data on their servers, or are messages removed once downloaded by the client(s) (remember that chat backups are performed via Google Drive/iCloud, not through some WhatApp-operated service)? If so, I'm not sure whether GDPR applies in the usual fashion.

(I'm on the same page with all of you: There should be a way to export full chat data.)

This is interesting. If you consider their web application, it should be retrieving the data from their web servers right? Plz let me know if I am wrong. So there should be some kind of msg storage available in their servers. Even we can go to old message history as far as it allows.

This is interesting. If you consider their web application, it should be retrieving the data from their web servers right? Plz let me know if I am wrong. So there should be some kind of msg storage available in their servers. Even we can go to old message history as far as it allows.

Web app retrieves messages from the phone, not the servers. At least in theory.

Okay, found the issue in HN, I tried to do something similar a while ago, my idea was to offer something like a SAAS CRM using Whatsapp, due to it's huge user base.

https://github.com/giovanialtelino/WhatsappMonitor

But it's pretty much impossible to officialy backup all your stuff, groups and chats are limited to 40k messages, more than that it will be lost.
Your rich media (images, audios, videos) have no official position, only date.
You can only backup one chat/group per time (no backup everything choice).
Your .txt backup has different date time format, it will consider your location, for example here in Brazil it's "dd/MM/yyyy HH:mm".

The main issue is lost message, and the need to backup every single chat, I personally have more than 100 chats, so a lot would be lost...
There are unnoficial tools like this one (https://github.com/B16f00t/whapa) that allow you to access your backup archive in the cloud and extract it, but at least for what I wanted to do it wasn't good enough, since it wasn't a official tool, but maybe it can help you guys.

Does WhatApp permanently keep any message data on their servers, or are messages removed once downloaded by the client(s) (remember that chat backups are performed via Google Drive/iCloud, not through some WhatApp-operated service)? If so, I'm not sure whether GDPR applies in the usual fashion.

So if I am correct, Whatsapp stores a key for your data, which itself is in the iCloud or Google Drive. So according to the GDPR Whatsapp must supply the key to me if i asked them to, because it is data which is related to me, isn't it?

I installed Whatsapp Business to have a autoresponder. I also completely disabled Notifications afterwards. So for a migration period everyone who writes me in whatsapp gets a small text where to find me. (Signal)

Okay, found the issue in HN, I tried to do something similar a while ago, my idea was to offer something like a SAAS CRM using Whatsapp, due to it's huge user base.

https://github.com/giovanialtelino/WhatsappMonitor

But it's pretty much impossible to officialy backup all your stuff, groups and chats are limited to 40k messages, more than that it will be lost.
Your rich media (images, audios, videos) have no official position, only date.
You can only backup one chat/group per time (no backup everything choice).
Your .txt backup has different date time format, it will consider your location, for example here in Brazil it's "dd/MM/yyyy HH:mm".

The main issue is lost message, and the need to backup every single chat, I personally have more than 100 chats, so a lot would be lost...
There are unnoficial tools like this one (https://github.com/B16f00t/whapa) that allow you to access your backup archive in the cloud and extract it, but at least for what I wanted to do it wasn't good enough, since it wasn't a official tool, but maybe it can help you guys.

Yeah but a tool to at least backup the most import stuff (and even if it is without media) is better than no tool at all. The problem with the date seems like the biggest issue to me, but this could be handled with a library too.

This is interesting. If you consider their web application, it should be retrieving the data from their web servers right? Plz let me know if I am wrong. So there should be some kind of msg storage available in their servers. Even we can go to old message history as far as it allows.

Web app retrieves messages from the phone, not the servers. At least in theory.

WhatsApp Web is also not usable if your phone is powered off or have no wifi connection.

I wonder, would a browser extension be a viable approach to exporting messages? I.e. open WhatsApp Web with your phone powered on, and let the extension scrape your data in a format you can import from Signal.

So using one of the reverse-engineered WhatsApp Web libraries, like go-whatsapp (or my own whatscl) lets you create a pretty complete backup of all of your conversations + media. (whatsxmpp uses this to populate its database with your group chat history, so it does indeed work.) That could be one avenue to look down if anyone was interested in implementing this feature...

Extracting data from WhatsApp is one part.
The problem is that there is no support for importing foreign chats into singal other than full restore from backup or read from SMS that works only for Android.
There is Signal for Android fork https://github.com/johanw666/Signal-Android by @johanw666 but it only support SMS format so it can't handle multi user conversation

On Android, WhatsApp supports writing a database backup to the shared storage. The backup contains all messages in an sqlite file "encrypted" by an AES key which is hardcoded into the application...

This is of course ridiculous from a security point of view (not unlike using the same RC4 keystream in both directions), but including that key in TextSecure might cause legal problems for this project.

If I do remember correctly then the key used to decrypt is found in WhatsApp app data directory and can be decrypted by an app, whose name I don't remember either

Doing it this way would lead to a loss of the included media, but the feature would help a lot of people to get rid of whatsapp and in the end it is better than nothing.

Agreed. I think the most important part is being able to port conversation history and existing groups.

On Android, WhatsApp supports writing a database backup to the shared storage. The backup contains all messages in an sqlite file "encrypted" by an AES key which is hardcoded into the application...
This is of course ridiculous from a security point of view (not unlike using the same RC4 keystream in both directions), but including that key in TextSecure might cause legal problems for this project.

If I do remember correctly then the key used to decrypt is found in WhatsApp app data directory and can be decrypted by an app, whose name I don't remember either

Yes, the private key to decrypt the backup is located in the local (sandboxed) app directory. If the device in question is rooted, one can obtain it easily, and parse the chat database with project like [0]. One trick is to move/transfer WhatsApp temporarily to old rooted Android and extract the private key and the encrypted backups.

If rooting is not an option, there is a method that involves spawning an android emulator (which is already "rooted"), install WhatsApp apk inside there, "transfer" the WhatsApp account temporarily, and then the private key can be extracted.

[0] https://github.com/B16f00t/whapa

I've been looking into this and have considered the following workflow:

1. pre-create groups i want to move over to signal from whatsapp
2. acquire and decrypt signal backup (using https://github.com/bepaald/signalbackup-tools)
3. acquire and decrypt whatsapp backup (using https://github.com/B16f00t/whapa)
4. merge chats from whatsapp db into signal db
5. re-encrpyt signal db and restore on phone

I've gotten partway through step 3 (trying to acquire whatsapp key), but having seen the signal database schema from step 2, I'm not confident...

3. acquire and decrypt whatsapp backup (using https://github.com/B16f00t/whapa)

I can't find a reliable way to do this on non-rooted Android devices. Aquiring the backup is not a problem, but how to retrieve the key for decryption?

Edit: I guess the answer is something like mentioned above (#1014 (comment)), but it's quite an effort to set this up and get it working. The last time I tried this using https://github.com/MarcoG3/WhatsDump it didn't succeed.

I thik we can do slightly different approach.
What's app allows exporting chat in txt from https://faq.whatsapp.com/android/chats/how-to-save-your-chat-history/?lang=en

It looks like this

dd.mm.yyyy, HH:MM - Author: text

e.g.:

21.12.2019, 14:00 - John Doe: OK 😃

Using this approach we should be able to easily migrate chats.

This can be confusing and maybe need a identifier in the txt file for the phone number of the other user

How about semi manual step. We can use chat export to get unencrypted backup of chats and then convert it into signal backup.

Alternatively there could be an import chat history option in Signal app (just like export in wahtsapp) so no third party app will be involved and this process could be done by non technical users.

This can be confusing and maybe need a identifier in the txt file for the phone number of the other user

Yes this might be a problem but as long and names in WhatsApp and Signal matches it should work.

We can add one confirmation step to allow user choose which contacts are in chat and match WhatsApp names with Signal IDs

I am user of neither but would like to take this opportunity to remind everyone of the importance of making sure that you can export your data out of a platform or a piece of software BEFORE you import/create your data into/on it. PG West 2010: The Barrier to Exit - Scott McNealy on the #1 Reason for Choosing Open Source Software + signal-export

Here's an idea. Maybe Syncthing (or perhaps a fork of Syncthing) could be the app between WhatsApp and Signal, doing a one-way sync of messages/media from WhatsApp to Signal. Syncthing has lots of great bells and whistles for securely sending data from one place to another. Including firewall punching.

I thik we can do slightly different approach.
What's app allows exporting chat in txt from https://faq.whatsapp.com/android/chats/how-to-save-your-chat-history/?lang=en

It looks like this

dd.mm.yyyy, HH:MM - Author: text

e.g.:

21.12.2019, 14:00 - John Doe: OK 😃

Using this approach we should be able to easily migrate chats.

The date format depends on user's phone settings. In india it was dd-mm-yyyy, here in US, it is mm-dd-yyyy from my phone.

Encryption on the messages looks might make this impossible.

I thik we can do slightly different approach.

What's app allows exporting chat in txt from https://faq.whatsapp.com/android/chats/how-to-save-your-chat-history/?lang=en

It looks like this

dd.mm.yyyy, HH:MM - Author: text

e.g.:

21.12.2019, 14:00 - John Doe: OK 😃

Using this approach we should be able to easily migrate chats.

The date format depends on user's phone settings. In india it was dd-mm-yyyy, here in US, it is mm-dd-yyyy from my phone.

Yeah, but there should be tonns of libraries out which could handle this, or not?

Encryption on the messages looks might make this impossible.

@reachjason The Chat Export feature gets around this without needing to decrypt chats.

Doing it this way would lead to a loss of the included media, but the feature would help a lot of people to get rid of whatsapp and in the end it is better than nothing.

@michi7801 At least in the UK (most likely everywhere), you can export WhatsApp chats with media. These get included like this:

DD/MM/YYYY, HH:MM - John Smith: IMG-20201114-WA0001.jpg (file attached)
<Media caption>

Exported media includes images (as JPEG), videos (as MP4), audio (as OPUS). I haven't tested GIFs, or uploaded files, but I can presume they work, too.

hey together
your dedication is great, but as stated in in CONTRIBUTING.md:
GitHub is not a generic discussion forum

because:
Bumping issues with a "+1" (or asking for updates) generates a lot of unnecessary email notifications and does not help anyone solve the issue any faster. Please be respectful of everyone's time and only comment when you have new information to add.

so you should open a thread on the community forum (or any other forum) to discuss your ideas

I created a working proof-of-concept fork that imports the msgstore.db into the Android Signal App. See (and potentially comment on this thread):
https://community.signalusers.org/t/migrate-from-whatsapp/10566/12?u=jukefoxer

According to Whatsapp FAQ

Media and messages you back up aren't protected by WhatsApp end-to-end encryption while in Google Drive.

Shouldn't we then be able to obtain the unencrypted msgstore.db from a google drive backup?

@nthacks Yes, but there's no easy way to access this Drive backup. A user would have to start a Google Takeout from Drive, as only WhatsApp itself can access this data.

Personally I think the important next step is to actually get an unencrypted and complete copy of all the data in WhatsApp, create a guide, and push it out via social media before the dataset are merged with Facebook.

We can figure out how to import later.

I just found out, that the export feature of Whatsapp will soon be available in Germany. This would make an import feature even more important!

@michi7801 You think a VPN would give me access to that feature, or would there be a separate version in Play Store?

With the newest WhatsApp update 2.21.11.3, chat export is available in Germany! 🎉

Examples:

[25.01.21, 15:10:00] Tom: This is a Message.
But it can include multi-line. 

[25.01.21, 15:10:00] Tom: <Anhang: 00000000-PHOTO-2021-01-25-15-10-00.jpg>

@tomxpcvx Could you please download the apk, and share it somewhere? I can't seem to actually get the apk even through VPN

@expelledboy
The feature should just pop up in the new versions of the German WhatsApp, no matter where you got it from. Just upgrade your WhatsApp to version 2.21.11.3

Telegram just got a feature to import Whatsapp chats https://telegram.org/blog/move-history/
Maybe someone can port this feature to Signal using the source code of the android telegram app?