9.5. Syntax Identifier¶
Identifiers specifies what to find
They are also called Character Classes
9.5.1. SetUp¶
>>> import re
9.5.2. Numeric¶
\d- digit\D- anything but digit
>>> TEXT = 'Mark Watney of Ares 3 landed on Mars on: Nov 7th, 2035 at 1:37 pm'
>>> re.findall(r'[0-9]', TEXT)
['3', '7', '2', '0', '3', '5', '1', '3', '7']
>>> re.findall(r'\d', TEXT)
['3', '7', '2', '0', '3', '5', '1', '3', '7']
>>> re.findall(r'\D', TEXT)
['M', 'a', 'r', 'k', ' ', 'W', 'a', 't', 'n', 'e', 'y', ' ', 'o', 'f',
' ', 'A', 'r', 'e', 's', ' ', ' ', 'l', 'a', 'n', 'd', 'e', 'd', ' ',
'o', 'n', ' ', 'M', 'a', 'r', 's', ' ', 'o', 'n', ':', ' ', 'N', 'o',
'v', ' ', 't', 'h', ',', ' ', ' ', 'a', 't', ' ', ':', ' ', 'p', 'm']
9.5.3. Whitespaces¶
\s- whitespace (space, tab, newline, non-breaking space)\S- anything but whitespace\n- newline\r\n- windows newline\r- carriage return\b- backspace\t- tab\v- vertical space\f- form feed
>>> TEXT = 'Mark Watney of Ares 3 landed on Mars on: Nov 7th, 2035 at 1:37 pm'
>>> re.findall(r'\s', TEXT)
[' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ']
>>> re.findall(r'\S', TEXT)
['M', 'a', 'r', 'k', 'W', 'a', 't', 'n', 'e', 'y', 'o', 'f', 'A', 'r',
'e', 's', '3', 'l', 'a', 'n', 'd', 'e', 'd', 'o', 'n', 'M', 'a', 'r',
's', 'o', 'n', ':', 'N', 'o', 'v', '7', 't', 'h', ',', '2', '0', '3',
'5', 'a', 't', '1', ':', '3', '7', 'p', 'm']
>>> re.findall(r'\n', TEXT)
[]
>>>
>>> re.findall(r'\r\n', TEXT)
[]
>>>
>>> re.findall(r'\r', TEXT)
[]
9.5.4. Anchors¶
Matches the empty string, but only at the beginning or end of a word
\b- word boundary\B- anything but word boundary
Examples:
\babc\b- performs a "whole words only" search
\Babc\B- pattern is fully surrounded by word characters
>>> TEXT = 'Mark Watney of Ares 3 landed on Mars on: Nov 7th, 2035 at 1:37 pm'
>>> re.findall(r'[a-z][a-z]', TEXT)
['ar', 'at', 'ne', 'of', 're', 'la', 'nd', 'ed', 'on', 'ar', 'on', 'ov', 'th', 'at', 'pm']
>>> re.findall(r'\b[a-z][a-z]\b', TEXT)
['of', 'on', 'on', 'at', 'pm']
>>> re.findall('\b[a-z][a-z]\b', TEXT) # without raw-string
[]
9.5.5. String¶
\w- any unicode alphabet character (lower or upper, also with diacritics (i.e. ąćęłńóśżź...), numbers and underscores\W- anything but any unicode alphabet character (i.e. whitespace, dots, comas, dashes)lowercase letters including diacritics (i.e. ąćęłńóśżź...) and accents
uppercase letters including diacritics (i.e. ąćęłńóśżź...) and accents
digits
underscores
_
Valid characters are the same as allowed in variable/modules names in Python:
>>> imie = 'Mark'
>>> IMIE = 'Mark'
>>> imię = 'Mark'
>>> imię1 = 'Mark'
>>> Imię_1 = 'Mark'
>>> TEXT = 'Mark Watney of Ares 3 landed on Mars on: Nov 7th, 2035 at 1:37 pm'
>>> re.findall(r'\w', TEXT)
['M', 'a', 'r', 'k', 'W', 'a', 't', 'n', 'e', 'y', 'o', 'f', 'A', 'r',
'e', 's', '3', 'l', 'a', 'n', 'd', 'e', 'd', 'o', 'n', 'M', 'a', 'r',
's', 'o', 'n', 'N', 'o', 'v', '7', 't', 'h', '2', '0', '3', '5', 'a',
't', '1', '3', '7', 'p', 'm']
>>> re.findall(r'\W', TEXT)
[' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ':', ' ', ' ', ',', ' ', ' ', ' ', ':', ' ']
Mind, that following code gives similar output to \w but it is not
completely true. \w would extract also unicode characters while this
[a-zA-Z0-9] will not.
>>> re.findall(r'[a-zA-Z0-9]', TEXT)
['M', 'a', 'r', 'k', 'W', 'a', 't', 'n', 'e', 'y', 'o', 'f', 'A', 'r',
'e', 's', '3', 'l', 'a', 'n', 'd', 'e', 'd', 'o', 'n', 'M', 'a', 'r',
's', 'o', 'n', 'N', 'o', 'v', '7', 't', 'h', '2', '0', '3', '5', 'a',
't', '1', '3', '7', 'p', 'm']
Example:
>>> text = 'cześć'
>>>
>>> re.findall(r'[a-z]', text)
['c', 'z', 'e']
>>>
>>> re.findall(r'\w', text)
['c', 'z', 'e', 'ś', 'ć']
>>>
>>> re.findall(r'\w', text, flags=re.ASCII)
['c', 'z', 'e']
>>>
>>> re.findall(r'\w', text, flags=re.UNICODE)
['c', 'z', 'e', 'ś', 'ć']
Flag re.UNICODE is set by default.
9.5.6. Use Case - 0x01¶
Phone
>>> phone = '+48 123 456 789'
>>> re.findall(r'\d', phone)
['4', '8', '1', '2', '3', '4', '5', '6', '7', '8', '9']
>>> phone = '+48 (12) 345 6789'
>>> re.findall(r'\d', phone)
['4', '8', '1', '2', '3', '4', '5', '6', '7', '8', '9']
9.5.7. Use Case - 0x02¶
Compare Phones
>>> PHONE1 = '+48 123 456 789'
>>> PHONE2 = '+48 (12) 345 6789'
>>>
>>> phone1 = re.findall(r'\d', PHONE1)
>>> phone2 = re.findall(r'\d', PHONE2)
>>>
>>> phone1 == phone2
True
9.5.8. Use Case - 0x03¶
EU VAT Tax ID
>>> number = '777-286-18-23'
>>> re.findall(r'\d', number)
['7', '7', '7', '2', '8', '6', '1', '8', '2', '3']
>>> number = '777-28-61-823'
>>> re.findall(r'\d', number)
['7', '7', '7', '2', '8', '6', '1', '8', '2', '3']
>>> number = '7772861823'
>>> re.findall(r'\d', number)
['7', '7', '7', '2', '8', '6', '1', '8', '2', '3']
9.5.9. Use Case - 0x04¶
Number and Spaces
>>> TEXT = 'Mark Watney of Ares 3 landed on Mars on: Nov 7th, 2035 at 1:37 pm'
>>> re.findall(r'[0-9]\s', TEXT)
['3 ', '5 ', '7 ']
>>> re.findall(r'\d\s', TEXT)
['3 ', '5 ', '7 ']
9.5.10. Assignments¶
"""
* Assignment: RE Syntax Identifier
* Complexity: easy
* Lines of code: 2 lines
* Time: 3 min
English:
1. Use regular expressions find in text:
a. all digits, use \d
b. unique digits
2. Run doctests - all must succeed
Polish:
1. Użyj wyrażeń regularnych wyszukiwania w tekście:
a. wszystkie cyfry, użyj \d
b. unikalne cyfry
2. Uruchom doctesty - wszystkie muszą się powieść
Hint:
* `re.findall()`
References:
[1] Authors: Wikipedia contributors
Title: Apollo 11
Publisher: Wikipedia
Year: 2019
Retrieved: 2019-12-14
URL: https://en.wikipedia.org/wiki/Apollo_11
Tests:
>>> import sys; sys.tracebacklimit = 0
>>> from pprint import pprint
>>> pprint(result_a, compact=True, width=72)
['1', '1', '2', '0', '1', '9', '6', '9', '2', '0', '1', '7', '6', '3',
'9', '2', '1', '1', '9', '6', '9', '0', '2', '5', '6', '1', '5', '1',
'9', '2', '3', '1', '4', '7', '5', '2', '1', '5', '2', '1', '3', '6']
>>> sorted(result_b)
['0', '1', '2', '3', '4', '5', '6', '7', '9']
"""
import re
TEXT = """Apollo 11 was the American spaceflight that first landed
humans on the Moon. Commander (CDR) Neil Armstrong and lunar module
pilot (LMP) Buzz Aldrin landed the Apollo Lunar Module (LM) Eagle on
July 20th, 1969 at 20:17 UTC, and Armstrong became the first person
to step (EVA) onto the Moon's surface (EVA) 6 hours 39 minutes later,
on July 21st, 1969 at 02:56:15 UTC. Aldrin joined him 19 minutes later.
They spent 2 hours 31 minutes exploring the site they had named Tranquility
Base upon landing. Armstrong and Aldrin collected 47.5 pounds (21.5 kg)
of lunar material to bring back to Earth as pilot Michael Collins (CMP)
flew the Command Module (CM) Columbia in lunar orbit, and were on the
Moon's surface for 21 hours 36 minutes before lifting off to rejoin
Columbia."""
# Find all digits in text, use \d
# Example: '1', '1', '2', '0', '1', '9', '6', ...
# type: list[str]
result_a = ...
# Find unique digits in text
# Example: '0', '1', '2', '3', '4', '5', '6', '7', '9'
# type: list[str]
result_b = ...
"""
* Assignment: RE Syntax Identifier
* Complexity: easy
* Lines of code: 2 lines
* Time: 3 min
English:
1. Use regular expressions find in text:
a. unique whitespace characters
b. unique non-whitespace characters
2. Run doctests - all must succeed
Polish:
1. Użyj wyrażeń regularnych wyszukiwania w tekście:
a. unikalne białe-znaki
b. unikalne nie-białe-znaki
2. Uruchom doctesty - wszystkie muszą się powieść
Hint:
* `re.findall()`
References:
[1] Authors: Wikipedia contributors
Title: Apollo 11
Publisher: Wikipedia
Year: 2019
Retrieved: 2019-12-14
URL: https://en.wikipedia.org/wiki/Apollo_11
Tests:
>>> import sys; sys.tracebacklimit = 0
>>> from pprint import pprint
>>> sorted(result_a)
['\\n', ' ']
>>> result = sorted(result_b)
>>> pprint(result, compact=True, width=72)
["'", '(', ')', ',', '.', '0', '1', '2', '3', '4', '5', '6', '7', '9',
':', 'A', 'B', 'C', 'D', 'E', 'J', 'L', 'M', 'N', 'P', 'R', 'T', 'U',
'V', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm',
'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'w', 'x', 'y', 'z']
"""
import re
TEXT = """Apollo 11 was the American spaceflight that first landed
humans on the Moon. Commander (CDR) Neil Armstrong and lunar module
pilot (LMP) Buzz Aldrin landed the Apollo Lunar Module (LM) Eagle on
July 20th, 1969 at 20:17 UTC, and Armstrong became the first person
to step (EVA) onto the Moon's surface (EVA) 6 hours 39 minutes later,
on July 21st, 1969 at 02:56:15 UTC. Aldrin joined him 19 minutes later.
They spent 2 hours 31 minutes exploring the site they had named Tranquility
Base upon landing. Armstrong and Aldrin collected 47.5 pounds (21.5 kg)
of lunar material to bring back to Earth as pilot Michael Collins (CMP)
flew the Command Module (CM) Columbia in lunar orbit, and were on the
Moon's surface for 21 hours 36 minutes before lifting off to rejoin
Columbia."""
# Find unique whitespace characters in text
# Example: '\n', ' '
# type: list[str]
result_a = ...
# Find unique non-whitespace characters in text
# Example: "'", '(', ')', ',', '.', '0', '1', '2', '3', '4', '5', '6', ...
# type: list[str]
result_b = ...
"""
* Assignment: RE Syntax Identifier
* Complexity: easy
* Lines of code: 3 lines
* Time: 5 min
English:
1. Use regular expressions find in text:
a. all word characters, use \w
b. unique word characters
c. unique non-word characters
2. Run doctests - all must succeed
Polish:
1. Użyj wyrażeń regularnych wyszukiwania w tekście:
a. znaki słów, użyj \w
b. znaki słów
c. nie-znaki słów
2. Uruchom doctesty - wszystkie muszą się powieść
Hint:
* `re.findall()`
References:
[1] Authors: Wikipedia contributors
Title: Apollo 11
Publisher: Wikipedia
Year: 2019
Retrieved: 2019-12-14
URL: https://en.wikipedia.org/wiki/Apollo_11
Tests:
>>> import sys; sys.tracebacklimit = 0
>>> from pprint import pprint
>>> pprint(result_a, compact=True, width=72)
['A', 'p', 'o', 'l', 'l', 'o', '1', '1', 'w', 'a', 's', 't', 'h', 'e',
'A', 'm', 'e', 'r', 'i', 'c', 'a', 'n', 's', 'p', 'a', 'c', 'e', 'f',
'l', 'i', 'g', 'h', 't', 't', 'h', 'a', 't', 'f', 'i', 'r', 's', 't',
'l', 'a', 'n', 'd', 'e', 'd', 'h', 'u', 'm', 'a', 'n', 's', 'o', 'n',
't', 'h', 'e', 'M', 'o', 'o', 'n', 'C', 'o', 'm', 'm', 'a', 'n', 'd',
'e', 'r', 'C', 'D', 'R', 'N', 'e', 'i', 'l', 'A', 'r', 'm', 's', 't',
'r', 'o', 'n', 'g', 'a', 'n', 'd', 'l', 'u', 'n', 'a', 'r', 'm', 'o',
'd', 'u', 'l', 'e', 'p', 'i', 'l', 'o', 't', 'L', 'M', 'P', 'B', 'u',
'z', 'z', 'A', 'l', 'd', 'r', 'i', 'n', 'l', 'a', 'n', 'd', 'e', 'd',
't', 'h', 'e', 'A', 'p', 'o', 'l', 'l', 'o', 'L', 'u', 'n', 'a', 'r',
'M', 'o', 'd', 'u', 'l', 'e', 'L', 'M', 'E', 'a', 'g', 'l', 'e', 'o',
'n', 'J', 'u', 'l', 'y', '2', '0', 't', 'h', '1', '9', '6', '9', 'a',
't', '2', '0', '1', '7', 'U', 'T', 'C', 'a', 'n', 'd', 'A', 'r', 'm',
's', 't', 'r', 'o', 'n', 'g', 'b', 'e', 'c', 'a', 'm', 'e', 't', 'h',
'e', 'f', 'i', 'r', 's', 't', 'p', 'e', 'r', 's', 'o', 'n', 't', 'o',
's', 't', 'e', 'p', 'E', 'V', 'A', 'o', 'n', 't', 'o', 't', 'h', 'e',
'M', 'o', 'o', 'n', 's', 's', 'u', 'r', 'f', 'a', 'c', 'e', 'E', 'V',
'A', '6', 'h', 'o', 'u', 'r', 's', '3', '9', 'm', 'i', 'n', 'u', 't',
'e', 's', 'l', 'a', 't', 'e', 'r', 'o', 'n', 'J', 'u', 'l', 'y', '2',
'1', 's', 't', '1', '9', '6', '9', 'a', 't', '0', '2', '5', '6', '1',
'5', 'U', 'T', 'C', 'A', 'l', 'd', 'r', 'i', 'n', 'j', 'o', 'i', 'n',
'e', 'd', 'h', 'i', 'm', '1', '9', 'm', 'i', 'n', 'u', 't', 'e', 's',
'l', 'a', 't', 'e', 'r', 'T', 'h', 'e', 'y', 's', 'p', 'e', 'n', 't',
'2', 'h', 'o', 'u', 'r', 's', '3', '1', 'm', 'i', 'n', 'u', 't', 'e',
's', 'e', 'x', 'p', 'l', 'o', 'r', 'i', 'n', 'g', 't', 'h', 'e', 's',
'i', 't', 'e', 't', 'h', 'e', 'y', 'h', 'a', 'd', 'n', 'a', 'm', 'e',
'd', 'T', 'r', 'a', 'n', 'q', 'u', 'i', 'l', 'i', 't', 'y', 'B', 'a',
's', 'e', 'u', 'p', 'o', 'n', 'l', 'a', 'n', 'd', 'i', 'n', 'g', 'A',
'r', 'm', 's', 't', 'r', 'o', 'n', 'g', 'a', 'n', 'd', 'A', 'l', 'd',
'r', 'i', 'n', 'c', 'o', 'l', 'l', 'e', 'c', 't', 'e', 'd', '4', '7',
'5', 'p', 'o', 'u', 'n', 'd', 's', '2', '1', '5', 'k', 'g', 'o', 'f',
'l', 'u', 'n', 'a', 'r', 'm', 'a', 't', 'e', 'r', 'i', 'a', 'l', 't',
'o', 'b', 'r', 'i', 'n', 'g', 'b', 'a', 'c', 'k', 't', 'o', 'E', 'a',
'r', 't', 'h', 'a', 's', 'p', 'i', 'l', 'o', 't', 'M', 'i', 'c', 'h',
'a', 'e', 'l', 'C', 'o', 'l', 'l', 'i', 'n', 's', 'C', 'M', 'P', 'f',
'l', 'e', 'w', 't', 'h', 'e', 'C', 'o', 'm', 'm', 'a', 'n', 'd', 'M',
'o', 'd', 'u', 'l', 'e', 'C', 'M', 'C', 'o', 'l', 'u', 'm', 'b', 'i',
'a', 'i', 'n', 'l', 'u', 'n', 'a', 'r', 'o', 'r', 'b', 'i', 't', 'a',
'n', 'd', 'w', 'e', 'r', 'e', 'o', 'n', 't', 'h', 'e', 'M', 'o', 'o',
'n', 's', 's', 'u', 'r', 'f', 'a', 'c', 'e', 'f', 'o', 'r', '2', '1',
'h', 'o', 'u', 'r', 's', '3', '6', 'm', 'i', 'n', 'u', 't', 'e', 's',
'b', 'e', 'f', 'o', 'r', 'e', 'l', 'i', 'f', 't', 'i', 'n', 'g', 'o',
'f', 'f', 't', 'o', 'r', 'e', 'j', 'o', 'i', 'n', 'C', 'o', 'l', 'u',
'm', 'b', 'i', 'a']
>>> pprint(sorted(result_b), compact=True, width=72)
['0', '1', '2', '3', '4', '5', '6', '7', '9', 'A', 'B', 'C', 'D', 'E',
'J', 'L', 'M', 'N', 'P', 'R', 'T', 'U', 'V', 'a', 'b', 'c', 'd', 'e',
'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's',
't', 'u', 'w', 'x', 'y', 'z']
>>> pprint(sorted(result_c), compact=True, width=72)
['\\n', ' ', "'", '(', ')', ',', '.', ':']
"""
import re
TEXT = """Apollo 11 was the American spaceflight that first landed
humans on the Moon. Commander (CDR) Neil Armstrong and lunar module
pilot (LMP) Buzz Aldrin landed the Apollo Lunar Module (LM) Eagle on
July 20th, 1969 at 20:17 UTC, and Armstrong became the first person
to step (EVA) onto the Moon's surface (EVA) 6 hours 39 minutes later,
on July 21st, 1969 at 02:56:15 UTC. Aldrin joined him 19 minutes later.
They spent 2 hours 31 minutes exploring the site they had named Tranquility
Base upon landing. Armstrong and Aldrin collected 47.5 pounds (21.5 kg)
of lunar material to bring back to Earth as pilot Michael Collins (CMP)
flew the Command Module (CM) Columbia in lunar orbit, and were on the
Moon's surface for 21 hours 36 minutes before lifting off to rejoin
Columbia."""
# Find all word characters in text, use \w
# Example: 'A', 'A', 'M', 'C', 'C', 'D', 'R', ...
# type: list[str]
result_a = ...
# Find unique word characters in text
# Example: 'g', '9', 'w', 'J', 'E', '5', 'f', ...
# type: list[str]
result_b = ...
# Find unique non-word characters in text
# Example: '\n', ' ', "'", '(', ')', ',', '.', ':'
# type: list[str]
result_c = ...
"""
* Assignment: RE Syntax Identifier
* Complexity: easy
* Lines of code: 5 lines
* Time: 5 min
English:
1. Use regular expressions find in text:
a. all two-letter conjunctives (two-letter words)
b. all three-letter conjunctives (three-letter words)
c. all conjunctives (both two- and three-letter words)
d. all acronyms (standalone words with two or three uppercase letters)
2. Run doctests - all must succeed
Polish:
1. Użyj wyrażeń regularnych wyszukiwania w tekście:
a. wszystkie dwu literowe spójniki (dwu-literowe słowa)
b. wszystkie trzy literowe spójniki (trzy literowe słow)
c. wszystkie spójniki (zarówno dwu jak i trzy literowe słowa)
d. wszystkie akronimy (słowa pisane dwoma lub trzemi dużymi literami)
2. Uruchom doctesty - wszystkie muszą się powieść
Hint:
* `re.findall()`
References:
[1] Authors: Wikipedia contributors
Title: Apollo 11
Publisher: Wikipedia
Year: 2019
Retrieved: 2019-12-14
URL: https://en.wikipedia.org/wiki/Apollo_11
Tests:
>>> import sys; sys.tracebacklimit = 0
>>> from pprint import pprint
>>> pprint(result_a, compact=True, width=72)
['on', 'on', 'at', 'to', 'on', 'at', 'kg', 'of', 'to', 'to', 'as', 'in',
'on', 'to']
>>> pprint(result_b, compact=True, width=72)
['was', 'the', 'the', 'and', 'the', 'and', 'the', 'the', 'him', 'the',
'had', 'and', 'the', 'and', 'the', 'for', 'off']
>>> pprint(result_c, compact=True, width=72)
['was', 'the', 'on', 'the', 'and', 'the', 'on', 'at', 'and', 'the',
'to', 'the', 'on', 'at', 'him', 'the', 'had', 'and', 'kg', 'of', 'to',
'to', 'as', 'the', 'in', 'and', 'on', 'the', 'for', 'off', 'to']
>>> pprint(result_d, compact=True, width=72)
['CDR', 'LMP', 'LM', 'UTC', 'EVA', 'EVA', 'UTC', 'CMP', 'CM']
"""
import re
TEXT = """Apollo 11 was the American spaceflight that first landed
humans on the Moon. Commander (CDR) Neil Armstrong and lunar module
pilot (LMP) Buzz Aldrin landed the Apollo Lunar Module (LM) Eagle on
July 20th, 1969 at 20:17 UTC, and Armstrong became the first person
to step (EVA) onto the Moon's surface (EVA) 6 hours 39 minutes later,
on July 21st, 1969 at 02:56:15 UTC. Aldrin joined him 19 minutes later.
They spent 2 hours 31 minutes exploring the site they had named Tranquility
Base upon landing. Armstrong and Aldrin collected 47.5 pounds (21.5 kg)
of lunar material to bring back to Earth as pilot Michael Collins (CMP)
flew the Command Module (CM) Columbia in lunar orbit, and were on the
Moon's surface for 21 hours 36 minutes before lifting off to rejoin
Columbia."""
# Find all two-letter conjunctives in text (two-letter words)
# Example: 'on', 'on', 'at', 'to', ...
# type: list[str]
result_a = ...
# Find all three-letter conjunctives in text (three-letter words)
# Example: 'was', 'the', 'the', 'and', ...
# type: list[str]
result_b = ...
# Find all conjunctives in text (both two- and three-letter words)
# Example: 'was', 'the', 'on', 'the', 'and', ...
# type: list[str]
result_c = ...
# Find all acronyms in text (standalone words with two or three uppercase letters)
# Example: 'CDR', 'LMP', 'LM', 'UTC', 'EVA', 'EVA', 'UTC', 'CMP', 'CM'
# type: list[str]
result_d = ...