Problem with folder name, over 256 character

[g-pane]

In order to get your bug addressed in a timely manner, or at all 😃, please fill out the below bug report. Please try to make it as easy as possible for us to understand what is going on. We may close out any bugs or issues without warning that are not complete or coherent.

In the bug template below anything is [square brackets] should be filled out or removed if the item doesn't apply.

Should you encounter an error that has not already been reported, please do the following when reporting it:
Bug Metadata

• Version of extract_msg: [x.x.x]
• Your python version: Python [3.6.7]
• How did you launch extract_msg?
  • My command line or
  • I used the extract_msg package

Describe the bug
It try to create a folder name with over 256 character

[ If applicable ]
**What code did you use or can we use to reproduce this error?

        if not os.path.isfile(msg_file):
            print(f"File non trovato: {msg_file}")
            return False
    
        os.makedirs(output_dir, exist_ok=True)
    
        msg = extract_msg.openMsg(msg_file)
        msg.save(customPath=output_dir, saveHeader=True, html=True)
        msg.close()
    
        print(f"File estratto correttamente in: {output_dir}")
        return True

Is there a message.msg file you want to share to help us reproduce this?

• Uploaded message (drag and drop on this window)
• Emailed message as an attachment to admins: [Enter Subject Line Here]

Traceback

Failed to create directory "C:\Users\XXX\Desktop\test\output_msg\2021-06-18_1155 R COMUNICAZIONE (3)\2021-06-18_1147 NAME EXAMPLE [TEST [TEST". Does it already exist?

Screenshots
[Insert any screenshots or debug pictures here]

Additional context
I'd appreciate a solution. Wouldn't it be better to use a safe GUID or add a parameter to specify a safe GUID folder? No one needs the folder name, since it's already included in the header.

P.S. A couple of questions

1. Is possibile to NOT include the "injectedHeader" inside the HTML? I'm removing with Python, i'd like a parameter to NOT add it
2. Is possible to embedded the image inside the HTML? Also this, i'm doing with Python

[TheElementalOfDestruction]

Square brackets are not, as far as I can tell, an illegal character to use in a folder name. I can create a folder name on windows (which given the file path appears to be what you are using) with only square brackets and it works fine. If it is illegal in your environment that is something I cannot find anyone else having an issue with.

For renaming the folder, the save function has a keyword argument customFilename which will be the name of the folder created which can be used to account for your system having unique criteria for what are illegal characters

For saving, it is generally meant to be relatively consistent, so there is no parameter to skip injecting the header. You will either need to implement your own save function, or open all files with a custom class that overrides the injectHtmlHeader method. The second would allow you to continue to use the save function. An example of what the overriden function would look like to skip the header is this:

def injectHtmlHeader(self, prepared: bool = False) -> bytes:
    return self.htmlBodyPrepared if prepared else self.htmlBody

preparedHtml = True is the option to inject images when using the save function

In addition to customFilename, there are also a few options to change what filename is used for the folder output. You can see details in the description for MessageBase.save

[g-pane]

Sorry, is not the square brackets, is too long the folder name: over 256 character.

I have try to use customFilename, but i have always the same error:
Failed to create directory "etc etc". Does it already exist?

msg.save(customPath=output_dir, saveHeader=True, html=True, customFilename='test')

I don't have understood how to override the injectHtmlHeader. Can you make both example? Thanks!

[TheElementalOfDestruction]

To be clear, the argument customFilename is going to be creating the folder and trying to append some numbers to it if it finds one that already exists. I might need to adjust the code to make it more reportive about what error it is actually getting, which can be done by changing extract_msg/msg_classes/message_base.py line 875 to the following two lines (for debugging):

except Exception as e:
    print(f'{e.__class__}: {e}')

By adjusting that a little bit it will at least allow us to figure out the exact nature of the error happening there. I'll put it on my todo list to try and have the reported error be more detailed in a full implementation.

For an example of injectHtmlHeader, you can subclass Message (or MessageBase) and have the entire class look like this:

from extract_msg.msg_classes.message_base import MessageBase

class MessageNoHeader(MessageBase):
    def injectHtmlHeader(self, prepared: bool = False) -> bytes:
        return self.htmlBodyPrepared if prepared else self.htmlBody

Then you simply replace msg = extract_msg.openMsg(msg_file) in your code with msg = MessageNoHeader(msg_file).

For the name being too long, there are already some protections in place to try to help with it, if I recall correctly, but they aren't extremely in depth due to the possible complexity meaning a proper solution may take more time than I currently have at the moment.

[g-pane]

This is the error:

<class 'OSError'>: [WinError 123] The syntax of the file, directory or volume name is incorrect: 'C:\Users\User\Desktop\test\output_msg\test\2021-06-18_1147 DELIVERY R CERTIFIED MAIL CERTIFIED MAIL R CERTIFIED MAIL CERTIFIED MAIL COMMUNICATION OF TRAVELING GENERATOR SETS PLEASE NOTE THAT THE PEC IS WITHOUT DOCUMENTATION. PARMA CUSTOMS OFFICE [DOGANEDGUD [DOGANEDGUD0068R'
Extraction error: Failed to create directory "C:\Users\User\Desktop\test\output_msg\test\2021-06-18_1147 DELIVERY R CERTIFIED MAIL CERTIFIED MAIL R CERTIFIED MAIL CERTIFIED MAIL COMMUNICATION OF TRAVELING GENERATOR SETS. PLEASE NOTE THAT THE PEC (certified email) DOES NOT HAVE DOCUMENTATION. PARMA CUSTOMS OFFICE [DOGANEDGUD [DOGANEDGUD0068R". Does it already exist?

The problem is the customFolder it create only the first one called "test". But if the mail have another mail inside it (like my case) the internal folder is called like the subject. Check this case for example:

As you can see, the internal folder is not called "test" but with the subject name. For this reason it doesn't work

Can you help me with subclassing for "remove" this bug? I need this thing to work, please man

import os
import re
import uuid
import base64
from datetime import datetime
from extract_msg.msg_classes.message_base import MessageBase

class openMsgCustomNoHeader(MessageBase):
    def injectHtmlHeader(self, prepared: bool = False) -> bytes:
        return self.htmlBodyPrepared if prepared else self.htmlBody

msg_file = r"C:\Users\Utente\Desktop\test\TestMsgToExtract.msg"
base_dir = os.path.dirname(msg_file)
output_dir = os.path.join(base_dir, "output_msg")

def estrai_msg(msg_file, output_dir):
    try:
        if not os.path.isfile(msg_file):
            print(f"File non trovato: {msg_file}")
            return False
    
        os.makedirs(output_dir, exist_ok=True)
    
        msg = openMsgCustomNoHeader(msg_file)
        msg.save(customPath=output_dir, saveHeader=True, html=True, customFilename='test')
        msg.close()
    
        print(f"File estratto correttamente in: {output_dir}")
        return True
    
    except Exception as e:
        print(f"Errore durante l'estrazione: {e}")
        return False

EDIT: I think it doesn't consider the overall lenght of the path but only of the single folder or subfolder

[TheElementalOfDestruction]

Ah, I forgot about embedded messages, so that will cause its own problems, yeah. The code that I thought I implemented to fix things looks like I never finished it, so I'll try to finish it up and create the necessary patch for you on Sunday. I just finished with the final for one of my classes and am celebrating a birthday tomorrow, so the day after will be the next time I'm free

[TheElementalOfDestruction]

So sorry it's taken me a minute to get back to this, I had to find enough free time where I could re-familiarize myself with that section of the code. Turns out I did actually implement the feature I was thinking of fixing, to some degree, with is that save accepts a maxNameLength keyword argument that can be used to limit the length of each folder name. It does not account for the total length of the path, so too much recursion can still cause problems, but that should hopefully help while I get to actually having something calculate the total length of the path and adjust for it

The default value for the keyword argument is 256 which is why the problems can still occur. I think I'll adjust it down for the next release to only 40 because that should be plenty for most cases, and if you need more for some reason you can just raise it.

Additionally, I'm now working on proper issue reporting for the "does it already exist" errors

[TheElementalOfDestruction]

Alright, I made all of the quick adjustments and pushed a fix for another issue into the new version (turns out I forgot to publish that fix, whoops). The recursive nature of some parts is probably going to make my job harder, but I'm next going to be working on getting the program to try to pre-calculate the length of the path in some way to see if an issue would occur, and try to reduce the length of names until that won't be a problem (if possible)

[g-pane]

You have not resolved the customFilename='test' that apply also to the embedded messages and not only to the main folder, right? I don't see that in the changelog.

Actually, i have replaced this part of your save with this, until you fixed it:

            if _zip:
                if raw:
                    raise IncompatibleOptionsError('The options `raw` and `zip` are incompatible.')
                if isinstance(_zip, (str, pathlib.Path)):
                    _zip = zipfile.ZipFile(_zip, 'a', zipfile.ZIP_DEFLATED)
                    kwargs['zip'] = _zip
                    createdZip = True
                path = pathlib.Path(kwargs.get('customPath', ''))
                _open = createZipOpen(_zip.open)
                mode = 'w'
            else:
                # HERE THE CHANGE!
                foldername = _custom_foldername()
                path = pathlib.Path(kwargs.get('customPath', '.')).absolute() / foldername
                mode = 'wb'
                _open = open
    
            kwargs['customFilename'] = None

And my custom filename function

def _custom_foldername():
        now = datetime.datetime.now().strftime('%Y%m%d_%H%M%S')
        random_numbers = ''.join([str(random.randint(0,9)) for _ in range(8)])
        return f"{now}_{random_numbers}"

Maybe it will not covert all the cases, but the folder name will be always the same lenght at least (also for embedded messages!) and always different using %Y%m%d_%H%M%S_8RANDOMNUMBERS. Can be smaller using also only numbers...
Waiting for your fix, thanks

[feljost]

Commenting to express my interest in also having a custom filename parameter in the save() method.

For example, if the email subject is cut off so that it ends with a blank space, this is an invalid folder name and we wont be able to use it. Setting the customPath will only set the current working directory, but recursively created folders wont work.

Here's an example file path it created for me

E:\sandbox\customPath_test\Talks about Product \Product1_Pitchdeck.pdf

[bigtonylewis]

Commenting to express my interest in also having a custom filename parameter in the save() method.

For example, if the email subject is cut off so that it ends with a blank space, this is an invalid folder name and we wont be able to use it. Setting the customPath will only set the current working directory, but recursively created folders wont work.

Here's an example file path it created for me

E:\sandbox\customPath_test\Talks about Product \Product1_Pitchdeck.pdf

I've created #471 for this