Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP #36

Draft
wants to merge 42 commits into
base: main
Choose a base branch
from
Draft

WIP #36

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
21f0432
Changed basic datastructure interfaces
kc611 Mar 26, 2023
e5e0821
Refactored renderer and simulator
kc611 Mar 26, 2023
e03688e
Refactored joins transformations and respective tests
kc611 Mar 27, 2023
d9f0b3c
Refactored tests and changes edge storage in SCFG to lists
kc611 Mar 27, 2023
fcb948d
Added precommit config
kc611 Mar 27, 2023
829ef61
Refactored loop and branch restructuring
kc611 Mar 28, 2023
5a00f5d
Fixed dictionary conversion testing
kc611 Mar 28, 2023
96b0391
Fixed failing tests in test_transforms
kc611 Mar 28, 2023
bf4a903
fix trailing whitespace and debugging leftovers
esc Mar 29, 2023
4f82268
Merge pull request #21 from kc611/block_name
esc Mar 29, 2023
779b0cb
refactor find_head_blocks
esc Mar 29, 2023
951a837
refactor find_branch_regions
esc Mar 29, 2023
4647610
refactor loop_restructure_helper
esc Mar 29, 2023
fe664b9
add typing and refactor NameGenerator
esc Mar 29, 2023
a57b137
Merge pull request #26 from esc/cosmetic_refactor_transformations
esc Mar 29, 2023
0ebc826
Merge pull request #27 from esc/typing_NameGenerator
esc Mar 29, 2023
390bc79
refactor find_headers_and_entries
esc Mar 29, 2023
8e0dd76
cross check trace against existing blocks
esc Mar 29, 2023
a5cfba2
fix simulator test bug found by tracing
esc Mar 29, 2023
e5c6b97
Merge pull request #28 from esc/refactor_find_entries_and_headers
kc611 Mar 30, 2023
733dd71
Merge pull request #30 from esc/add_tracing_to_sim_tests
kc611 Mar 30, 2023
129b6e0
fixup formatting and typos in README
esc Mar 30, 2023
58054c3
fix two bugs leftover from refactoring
esc Mar 31, 2023
c0cf672
Merge pull request #31 from esc/fixup_readme
kc611 Mar 31, 2023
167b552
Merge pull request #32 from esc/fix_main
kc611 Mar 31, 2023
c9ffdbb
Merge pull request #25 from kc611/pre_commit
esc Mar 31, 2023
4b28dc5
refactor find_exiting_and_exits
esc Mar 31, 2023
2a59a96
Merge pull request #33 from esc/refactor_find_exiting_and_exits
kc611 Apr 3, 2023
6c7e045
Added region heirarchy and fixed region rendering
kc611 Mar 29, 2023
b3e71aa
Fixed failing tests regarding string representation
kc611 Mar 29, 2023
7557526
Removed scfg from regions
kc611 Mar 29, 2023
f184a81
Added dummy Region within SCFG graphs
kc611 Mar 31, 2023
7db364b
Fixed Rendering
kc611 Mar 31, 2023
cc0e95a
Added tests for loops
kc611 Apr 3, 2023
a86040f
Fixed review comments
kc611 Apr 3, 2023
6174bab
Merge pull request #29 from kc611/regions
esc Apr 3, 2023
66729df
fix set intialization in region iterator
esc Apr 4, 2023
dd17688
fix conversion of out_targets in implicit jumps
esc Apr 4, 2023
1dce90f
Merge pull request #34 from esc/fix_set_init
kc611 Apr 4, 2023
647a444
Merge pull request #35 from esc/fix_python_ingestion
kc611 Apr 4, 2023
c750c2a
WIP
esc Apr 4, 2023
53f6206
fix stackoverflow caused by recursion
esc Apr 4, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
exclude: |
(?x)^(
docs/.*
)$
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
hooks:
- id: debug-statements
- id: check-merge-conflict
- repo: https://github.com/asottile/pyupgrade
rev: v3.3.1
hooks:
- id: pyupgrade
args: ["--py38-plus"]
- repo: https://github.com/psf/black
rev: 23.1.0
hooks:
- id: black
language_version: python3
- repo: https://github.com/pycqa/flake8
rev: 6.0.0
hooks:
- id: flake8
- repo: https://github.com/pycqa/isort
rev: 5.11.5
hooks:
- id: isort
- repo: https://github.com/humitos/mirrors-autoflake.git
rev: v1.1
hooks:
- id: autoflake
args: ['--in-place', '--remove-all-unused-imports', '--remove-unused-variable']
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.1.1
hooks:
- id: mypy
additional_dependencies:
- types-filelock
- types-setuptools
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ within Numba.
The code in this repository is an implementation of the CFG restructuring
algorithms in Bahmann2015, specifically those from section 4.1 and 4.2: namely
"loop restructuring" and "branch restructuring". These are interesting for
Numba because they serve to clearly identify regions withing the Python
Numba because they serve to clearly identify regions within the Python
bytecode.

## dependencies
Expand Down Expand Up @@ -42,16 +42,16 @@ numba_rvsdg
├── core
│   ├── datastructures
│   │   ├── basic_block.py # BasicBlock implementation
│   │   ├── block_map.py # BlockMap implementation, maps labels to blocks
│   │   ├── byte_flow.py # ByteFlow implementation, BlockMap + bytecode
│   │   ├── scfg.py # SCFG implementation, maps labels to blocks
│   │   ├── byte_flow.py # ByteFlow implementation, SCFG + bytecode
│   │   ├── flow_info.py # Converts program to ByteFlow
│   │   └── labels.py # Collection of Label classes
│   ├── transformations.py # Algorithms
│   └── utils.py # Miscellaneous utilities
├── networkx_vendored
│   └── scc.py # Strongly Connected Componets (loop detection)
├── rendering
│   └── rendering.py # Graphivz based rendering of BlockMaps
│   └── rendering.py # Graphivz based rendering of SCFGs
├── tests
│   ├── simulator.py # Simulator utility for running SCFGs
│   ├── test_byteflow.py # Testung ByteFlow and others
Expand Down
107 changes: 36 additions & 71 deletions numba_rvsdg/core/datastructures/basic_block.py
Original file line number Diff line number Diff line change
@@ -1,47 +1,31 @@
import dis
from collections import ChainMap
from typing import Tuple, Dict, List
from dataclasses import dataclass, field, replace
from typing import Dict, List
from dataclasses import dataclass, field, InitVar

from numba_rvsdg.core.datastructures.labels import Label
from numba_rvsdg.core.datastructures.labels import Label, NameGenerator, BlockName
from numba_rvsdg.core.utils import _next_inst_offset


@dataclass(frozen=True)
class BasicBlock:
label: Label
"""The corresponding Label for this block. """

_jump_targets: Tuple[Label] = tuple()
"""Jump targets (branch destinations) for this block"""

backedges: Tuple[Label] = tuple()
"""Backedges for this block."""
class Block:
name_gen: InitVar[NameGenerator]
"""Block Name Generator associated with this Block.
Note: This is an initialization only argument and not
a class attribute."""

@property
def is_exiting(self) -> bool:
return not self.jump_targets
label: Label
"""The corresponding Label for this block."""

@property
def fallthrough(self) -> bool:
return len(self._jump_targets) == 1

@property
def jump_targets(self) -> Tuple[Label]:
acc = []
for j in self._jump_targets:
if j not in self.backedges:
acc.append(j)
return tuple(acc)
@dataclass(frozen=True)
class BasicBlock(Block):

def replace_backedge(self, target: Label) -> "BasicBlock":
if target in self.jump_targets:
assert not self.backedges
return replace(self, backedges=(target,))
return self
block_name: BlockName = field(init=False)
"""Unique name identifier for this block"""

def replace_jump_targets(self, jump_targets: Tuple) -> "BasicBlock":
return replace(self, _jump_targets=jump_targets)
def __post_init__(self, name_gen: NameGenerator):
block_name = name_gen.new_block_name(label=self.label)
object.__setattr__(self, "block_name", block_name)


@dataclass(frozen=True)
Expand Down Expand Up @@ -83,44 +67,25 @@ class BranchBlock(BasicBlock):
variable: str = None
branch_value_table: dict = None

def replace_jump_targets(self, jump_targets: Tuple) -> "BasicBlock":
fallthrough = len(jump_targets) == 1
old_branch_value_table = self.branch_value_table
new_branch_value_table = {}
for target in self.jump_targets:
if target not in jump_targets:
# ASSUMPTION: only one jump_target is being updated
diff = set(jump_targets).difference(self.jump_targets)
assert len(diff) == 1
new_target = next(iter(diff))
for k, v in old_branch_value_table.items():
if v == target:
new_branch_value_table[k] = new_target
else:
# copy all old values
for k, v in old_branch_value_table.items():
if v == target:
new_branch_value_table[k] = v

return replace(
self,
_jump_targets=jump_targets,
branch_value_table=new_branch_value_table,
)

# Maybe we can register new blocks over here instead of static lists
block_types = {
"basic": BasicBlock,
"python_bytecode": PythonBytecodeBlock,
"control_variable": ControlVariableBlock,
"branch": BranchBlock,
}

@dataclass(frozen=True)
class RegionBlock(BasicBlock):
kind: str = None
headers: Dict[Label, BasicBlock] = None
"""The header of the region"""
subregion: "BlockMap" = None
"""The subgraph excluding the headers
"""
exit: Label = None
"""The exit node.
"""

def get_full_graph(self):
graph = ChainMap(self.subregion.graph, self.headers)
return graph
def get_block_class(block_type_string: str):
if block_type_string in block_types:
return block_types[block_type_string]
else:
raise TypeError(f"Block Type {block_type_string} not recognized.")

def get_block_class_str(basic_block: BasicBlock):
for key, value in block_types.items():
if isinstance(basic_block, value):
return key
else:
raise TypeError(f"Block Type of {basic_block} not recognized.")
Loading