RE parsing principles

RegEx patterns operate on two types of atoms:

characters
like ASCII characters, e.g. a, B, -, \n
betweeners
places between other characters, e.g. \b matches between a word and a space

Very important is to know that betweeners are optional to find the match. The pattern yellow\b \band as long as pattern yellow and will match the same strings.

The ^ and $ are betweeners, not characters. How they work can be little mysterious. For sure $ doesn't match \n (the new line character). "Say what?" you ask:)! When the re.MULTILINE flag is off, ^ matches the beginning of a string and $ the end of it. That means $ matches very often after \n character if it's the last character in a string. Which is very common at the end of text files.

Lets take this string: \nMango is yellow.\nBanana is rip.\n The pattern ^ matches at the beginning: I\nMango is yellow.\nBanana is rip.\n Try it yourself and $ at the very end: \nMango is yellow.\nBanana is rip.\nI Try it yourself But that's a lie. The $ matches two times in the previous example. I wanted it looks like that only for teaching purposes... Sorry. Please go to the Traps section for further explanation. Lets now assume it matches just once, at the very end.

When re.MULTILINE is on, the ^ matches additionally after every \n and $ before every \n. So for last string, the ^ pattern matches 4 times: I\nIMango is yellow.\nIBanana is rip.\nI Try it yourself and the $ also 4 times: I\nMango is yellow.I\nBanana is rip.I\nI Try it yourself

One more important note. RegEx always looks for a match in the whole string. The re.MULTILINE flag doesn't stop matching after end of the line. It just changes meaning of the ^ and $ betweeners.

Another flag re.DOTALL changes . (the dot) special character. By default it matches everything except \n. That can be misleading for beginners, suggesting that RegEx works with lines separately. When re.DOTALL is on, . (the dot) matches alsothe \n character, so absolutely everything in a string.

As a misleading example lets consider the previous text again. The ^.*$ pattern will not match anything if flags re.DOTALL and re.MULTILINE are off! Even it looks like it should match whole string not matter what! \nMango is yellow.\nBanana is rip.\n Try it yourself

The \A betweener matches only the beginning of a string. It is not equal to the ^ character because the flag re.MULTILINE doesn't modify it's meaning. It always matches only beginnings of strings, not beginnings of lines. The same applies to the \Z betweener compared with the $.