chimcla.stage_2a_bar_selection

Important

Historically grown and messy module for bar selection (or “separation”)

• contains:

  • class CavityCarrierImageAnalyzer (“FormenTrägerBildAnalyzer”)

  • class ExtendedSqliteDict

Module Contents

Classes

ExtendedSqliteDict
Attr_Array	Special layer on top of numpy arrays which accept custom attributes
Fitter
CavityCarrierImageAnalyzer
AngleAnalyzer
HistEvaluation	HistogramEvaluation for one complete (81-cell) image

Functions

unpack_key
load_img
rgb
mpl_draw_rect
background	decorator for parallelization
put_text
vertical_detrend	Identify a linear trend in vertical direction and compensate it.
get_bbox_list_robust
get_bbox_list
assign_row_col	problem: The list of bounding boxes is not sorted. it has to be calculated in which row an col every bb is. Also after this function the bbox_list is sorted (starting with 1st row)
index_combinations
find_missing_boxes	Iterate through a list of (extended) bounding boxes
handle_missing_boxes
get_img_and_bbox_list
get_raw_cell
rotate_img
add_padding
_rotate_img
piecewise_linear2
piecewise_linear3
process_column
get_test_column_idcs	Support for angle detection of a bar: find out which columns are relevant to test
get_angle
correct_angle
gaussian_kernel
gaussian_smooth
symlog_transform
get_symlog_hist	:param dc: debug container
get_symlog_hist_from_cell	:param delta: offset in pixels which will be ignored at each border
convert_to_dict
select_bar_from_file
get_angle_from_hough
adapt_rotation_and_margin
main
get_angle_from_moments
get_border_columns	For a given cell image return the indices of columns, that are the border of the chocolate bar
analyze_img_fpath	Convert a path into a pfname-object (with attributes like .date, .time, …)
get_cell_key_from_fpath
get_img_list
get_original_image_fpath
get_hist_for_cell_pict
bright
bgreen
bred
yellow

Data

REPO_ROOT
DATA_DIR
BBOX_EXPECTED_WITH
BBOX_EXPECTED_HEIGHT
BBOX_TOL
PREPROCESS_BORDERS
BBOX_MIN_WITH
CRIT_PIX_THRESHOLD
vv
CELL_KEY_END
cell_keys
uk
db
img_bbox_cache
colors
BAR_THRESHOLD
CavityCarrierImageAnalyzier
aa
pfname

API

chimcla.stage_2a_bar_selection.REPO_ROOT

‘join(…)’

chimcla.stage_2a_bar_selection.DATA_DIR

‘join(…)’

chimcla.stage_2a_bar_selection.BBOX_EXPECTED_WITH

26

chimcla.stage_2a_bar_selection.BBOX_EXPECTED_HEIGHT

104

chimcla.stage_2a_bar_selection.BBOX_TOL

6

chimcla.stage_2a_bar_selection.PREPROCESS_BORDERS

False

chimcla.stage_2a_bar_selection.BBOX_MIN_WITH

23

chimcla.stage_2a_bar_selection.CRIT_PIX_THRESHOLD

5

chimcla.stage_2a_bar_selection.vv

None

chimcla.stage_2a_bar_selection.CELL_KEY_END

None

chimcla.stage_2a_bar_selection.cell_keys

None

chimcla.stage_2a_bar_selection.unpack_key(txt: str)

chimcla.stage_2a_bar_selection.uk

None

class chimcla.stage_2a_bar_selection.ExtendedSqliteDict(filename=None, tablename='unnamed', flag='c', autocommit=False, journal_mode='DELETE', encode=encode, decode=decode, encode_key=identity, decode_key=identity, timeout=5, outer_stack=True)

Bases: sqlitedict.SqliteDict

put(main_key, sub_key, value, commit=False)

put_container(main_key, cont, key_list, commit=False)

convenience function to write selected content of a container to the database

VALID_FLAGS

[‘c’, ‘r’, ‘w’, ‘n’]

_new_conn()

__enter__()

__exit__(*exc_info)

__str__()

__repr__()

__len__()

__bool__()

iterkeys()

itervalues()

iteritems()

keys()

values()

items()

__contains__(key)

__getitem__(key)

__setitem__(key, value)

__delitem__(key)

update(items=(), **kwds)

__iter__()

clear()

static get_tablenames(filename)

commit(blocking=True)

sync

None

close(do_log=True, force=False)

terminate()

__del__()

get(key, default=None)

__or__(other)

__ror__(other)

__ior__(other)

__copy__()

copy()

classmethod fromkeys(iterable, value=None)

__slots__

()

__marker

‘object(…)’

pop(key, default=__marker)

popitem()

setdefault(key, default=None)

__abc_tpflags__

None

__eq__(other)

__reversed__

None

classmethod __subclasshook__(C)

__class_getitem__

‘classmethod(…)’

chimcla.stage_2a_bar_selection.db

‘ExtendedSqliteDict(…)’

chimcla.stage_2a_bar_selection.load_img(fpath, rgb=False)

chimcla.stage_2a_bar_selection.rgb(img)

chimcla.stage_2a_bar_selection.mpl_draw_rect(x, y, w, h, ax=None, linewidth=1, edgecolor='r', facecolor='none')

chimcla.stage_2a_bar_selection.background(f)

decorator for parallelization

chimcla.stage_2a_bar_selection.put_text(image, origin, text, fontScale=0.35, color=(255, 0, 0), thickness=1)

chimcla.stage_2a_bar_selection.vertical_detrend(img, y_start, y_end, dc=None)

Identify a linear trend in vertical direction and compensate it.

Intended to be used for bbox detection only

:param X1: specify region of interest :param X2: specify region of interest :param Y1: specify region of interest :param Y2: specify region of interest :param y_start: start of trend detection :param y_end: end of trend detection

chimcla.stage_2a_bar_selection.get_bbox_list_robust(img, expected_number, plot=False, return_all=False, dc=None)

chimcla.stage_2a_bar_selection.get_bbox_list(img, plot=False, return_all=False, thresh=75, dc=None)

chimcla.stage_2a_bar_selection.assign_row_col(bbox_list)

problem: The list of bounding boxes is not sorted. it has to be calculated in which row an col every bb is. Also after this function the bbox_list is sorted (starting with 1st row)

chimcla.stage_2a_bar_selection.index_combinations()

chimcla.stage_2a_bar_selection.find_missing_boxes(bbox_list)

Iterate through a list of (extended) bounding boxes

chimcla.stage_2a_bar_selection.handle_missing_boxes(bbox_list, fpath)

chimcla.stage_2a_bar_selection.img_bbox_cache

None

chimcla.stage_2a_bar_selection.get_img_and_bbox_list(fpath)

chimcla.stage_2a_bar_selection.get_raw_cell(fpath, hr_row, hr_col, e=0, f=0, plot=False)

chimcla.stage_2a_bar_selection.colors

[‘#1f77b4’, ‘#ff7f0e’, ‘#2ca02c’, ‘#d62728’, ‘#9467bd’, ‘#8c564b’, ‘#e377c2’, ‘#7f7f7f’, ‘#bcbd22’, …

class chimcla.stage_2a_bar_selection.Attr_Array(shape, dtype=float, buffer=None, offset=0, strides=None, order=None)

Bases: numpy.ndarray

Special layer on top of numpy arrays which accept custom attributes

Initialization

__new__(input_array, **kwargs)

__abs__()

__add__(value)

__and__(value)

__array__(dtype=None)

__array_wrap__(obj)

__contains__(key)

__copy__()

__deepcopy__(memo)

__divmod__(value)

__eq__(value)

__float__()

__floordiv__()

__ge__(value)

__getitem__(key)

__gt__(value)

__iadd__(value)

__iand__(value)

__ifloordiv__(value)

__ilshift__(value)

__imod__(value)

__imul__(value)

__int__()

__invert__()

__ior__(value)

__ipow__(value)

__irshift__(value)

__isub__(value)

__itruediv__(value)

__ixor__(value)

__le__(value)

__len__()

__lshift__(value)

__lt__(value)

__matmul__(value)

__mod__(value)

__mul__(value)

__ne__(value)

__neg__()

__or__(value)

__pos__()

__pow__()

__repr__()

__rshift__()

__setitem__(key, value)

__str__()

__sub__(value)

__truediv__(value)

__xor__(value)

all(axis=None, out=None, keepdims=False)

any(axis=None, out=None, keepdims=False)

argmax(axis=None, out=None)

argmin(axis=None, out=None)

argpartition(kth, axis=-1, kind='introselect', order=None)

argsort(axis=-1, kind='quicksort', order=None)

astype(dtype, order='K', casting='unsafe', subok=True, copy=True)

byteswap(inplace=False)

choose(choices, out=None, mode='raise')

clip(min=None, max=None, out=None)

compress(condition, axis=None, out=None)

conj()

conjugate()

copy(order='C')

cumprod(axis=None, dtype=None, out=None)

cumsum(axis=None, dtype=None, out=None)

diagonal(offset=0, axis1=0, axis2=1)

dot(b, out=None)

dump(file)

dumps()

fill(value)

flatten(order='C')

getfield(dtype, offset=0)

item(*args)

itemset(*args)

max(axis=None, out=None)

mean(axis=None, dtype=None, out=None, keepdims=False)

min(axis=None, out=None, keepdims=False)

newbyteorder(new_order='S')

nonzero()

partition(kth, axis=-1, kind='introselect', order=None)

prod(axis=None, dtype=None, out=None, keepdims=False)

ptp(axis=None, out=None)

put(indices, values, mode='raise')

ravel(order='C')

repeat(repeats, axis=None)

reshape(shape, order='C')

resize(new_shape, refcheck=True)

round(decimals=0, out=None)

searchsorted(v, side='left', sorter=None)

setfield(val, dtype, offset=0)

setflags(write=None, align=None, uic=None)

sort(axis=-1, kind='quicksort', order=None)

squeeze(axis=None)

std(axis=None, dtype=None, out=None, ddof=0, keepdims=False)

sum(axis=None, dtype=None, out=None, keepdims=False)

swapaxes(axis1, axis2)

take(indices, axis=None, out=None, mode='raise')

tobytes(order='C')

tofile(fid, sep='', format='%s')

tolist()

tostring(order='C')

trace(offset=0, axis1=0, axis2=1, dtype=None, out=None)

transpose(*axes)

var(axis=None, dtype=None, out=None, ddof=0, keepdims=False)

view(dtype=None, type=None)

classmethod __class_getitem__(value)

chimcla.stage_2a_bar_selection.rotate_img(img, angle, border_value=255, padding=3)

chimcla.stage_2a_bar_selection.add_padding(img, padding=3)

chimcla.stage_2a_bar_selection._rotate_img(img, angle, border_value=255)

chimcla.stage_2a_bar_selection.piecewise_linear2(x, x0, y0, k1, k2)

chimcla.stage_2a_bar_selection.piecewise_linear3(x, x0, y0, x1, k1, k2, k3)

class chimcla.stage_2a_bar_selection.Fitter(ii)

Initialization

fit_sequence2(seq, p0=None)

fit_sequence3(seq, p0=None)

smart_fit_sequence3(seq)

pw_err(model, params, seq)

chimcla.stage_2a_bar_selection.process_column(img, j, plot=False)

chimcla.stage_2a_bar_selection.BAR_THRESHOLD

110

chimcla.stage_2a_bar_selection.get_test_column_idcs(img, plot=False)

Support for angle detection of a bar: find out which columns are relevant to test

chimcla.stage_2a_bar_selection.get_angle(img, dc=None)

chimcla.stage_2a_bar_selection.correct_angle(img, y_offset=5, dc=None)

chimcla.stage_2a_bar_selection.gaussian_kernel(size, sigma)

chimcla.stage_2a_bar_selection.gaussian_smooth(data, kernel_size, sigma)

chimcla.stage_2a_bar_selection.symlog_transform(x, linthresh)

exception chimcla.stage_2a_bar_selection.InconsistentAngle

Bases: ValueError

class __cause__

class __context__

__delattr__()

__dir__()

__eq__()

__format__()

__ge__()

__getattribute__()

__getstate__()

__gt__()

__hash__()

__le__()

__lt__()

__ne__()

__new__()

__reduce__()

__reduce_ex__()

__repr__()

__setattr__()

__setstate__()

__sizeof__()

__str__()

__subclasshook__()

class __suppress_context__

class __traceback__

add_note()

class args

with_traceback()

exception chimcla.stage_2a_bar_selection.MissingBoundingBoxes

Bases: ValueError

class __cause__

class __context__

__delattr__()

__dir__()

__eq__()

__format__()

__ge__()

__getattribute__()

__getstate__()

__gt__()

__hash__()

__le__()

__lt__()

__ne__()

__new__()

__reduce__()

__reduce_ex__()

__repr__()

__setattr__()

__setstate__()

__sizeof__()

__str__()

__subclasshook__()

class __suppress_context__

class __traceback__

add_note()

class args

with_traceback()

chimcla.stage_2a_bar_selection.get_symlog_hist(img_fpath, hr_row, hr_col, delta=None, return_cell=False, dc=None, ccia=None)

:param dc: debug container

chimcla.stage_2a_bar_selection.get_symlog_hist_from_cell(cell, delta=None, dc=None)

:param delta: offset in pixels which will be ignored at each border

chimcla.stage_2a_bar_selection.convert_to_dict(bbox_list, img, desired_idx_pair=None)

chimcla.stage_2a_bar_selection.select_bar_from_file(fpath, hr_row, hr_col)

chimcla.stage_2a_bar_selection.get_angle_from_hough(cell)

chimcla.stage_2a_bar_selection.adapt_rotation_and_margin(bbox, img, forced_angle=None, plot=True)

chimcla.stage_2a_bar_selection.main()

class chimcla.stage_2a_bar_selection.CavityCarrierImageAnalyzer(img_fpath, bboxes=True)

Initialization

BBOX_EXPECTED_WITH

26

BBOX_EXPECTED_HEIGHT

104

BBOX_EXPECTED_DX

7

BBOX_EXPECTED_DY

51

BBOX_ROWS

3

BBOX_COLS

27

BBOX_NUMBER

None

ROI_DX

5

ROI_DY

5

THRESHOLDS

[75, 70, 80, 65, 85]

show(ax=None)

static replace_cell(img, bbox, cell_img)

detrend_upper_row()

Identify a linear trend in the upper row and compensate it.

Intended to be used for bbox detection only

make_sorted_bbox_list(plot=False)

handle_missing_boxes()

get_guessed_bbox_roi(bbox_known, col_known, col_missing)

find_bbox_border(bb_container)

handle_missing_box(bb_container, row, col)

get_bbox_for_cell(hr_row, hr_col)

get_raw_cell(hr_row, hr_col, e=0, f=0, rgb=False, plot=False, uncorrected=False)

find_cell_corners(hr_row, hr_col, plot=False, dc=None)

return absolute coordinates of the upper left corner of a cell

fill_corners_dict(plot=None, dc=None)

get_horizontal_line(hr_row, corner_name='upper_left', plot=False)

get_bbox_based_angle(hr_row, hr_col)

get_angle_from_cell(hr_row, hr_col, e=3, f=3, dc=None)

estimate_angle_for_cell(hr_row, hr_col, e=3, f=3, dc=None)

Use angle analyzer if possible. Evaluate image otherwise

get_corrected_cell(hr_row, hr_col, e=3, f=3, cut_to_bb=True, plot=False, force_angle=None, dc=None)

_get_corrected_cell(hr_row, hr_col, e=3, f=3, cut_to_bb=True, plot=False, force_angle=None, dc=None)

chimcla.stage_2a_bar_selection.CavityCarrierImageAnalyzier

None

class chimcla.stage_2a_bar_selection.AngleAnalyzer

Initialization

load_data()

process_dict(hist_dict, alpha=0.5, plot=False)

process_all_dicts(alpha=0.5, plot=False)

fit_curves(plot=False)

get_angle_offset_for_img(ccia: chimcla.stage_2a_bar_selection.CavityCarrierImageAnalyzer)

get_angle_offset_for_img2(img_angles)

check_offset()

chimcla.stage_2a_bar_selection.aa

‘AngleAnalyzer(…)’

chimcla.stage_2a_bar_selection.get_angle_from_moments(img)

chimcla.stage_2a_bar_selection.get_border_columns(cell_img, dark_value_thresh=100, dark_share_thresh=0.7, dc=None)

For a given cell image return the indices of columns, that are the border of the chocolate bar

chimcla.stage_2a_bar_selection.pfname

‘namedtuple(…)’

chimcla.stage_2a_bar_selection.analyze_img_fpath(fpath) → chimcla.stage_2a_bar_selection.pfname

Convert a path into a pfname-object (with attributes like .date, .time, …)

chimcla.stage_2a_bar_selection.get_cell_key_from_fpath(fpath)

chimcla.stage_2a_bar_selection.get_img_list(img_dir, limit=None)

chimcla.stage_2a_bar_selection.get_original_image_fpath(img_fpath, cropped=True, resized=True) → str

chimcla.stage_2a_bar_selection.get_hist_for_cell_pict(fpath)

class chimcla.stage_2a_bar_selection.HistEvaluation(img_fpath: str, suffix: str = '', ev_crit_pix=False, training_data_flag=False, history_eval_flag=False)

HistogramEvaluation for one complete (81-cell) image

Initialization

:param ev_crit_pix: bool; default False; evaluate critical pixels if true additional information about the critical pixels is collected and stored :param training_data_flag: bool; save plots also for non-critical cells + store plots for raw cell images

hist_dict_path

‘join(…)’

total_res_fpath

None

CS_LIMIT

40

get_criticality_summary(save_to_db=False)

iterate over self.criticality_container_cache and return cumulated values

get_bbox_for_cell(hr_row, hr_col)

initialize_hist_cache()

static get_result_files(result_dir)

static _get_result_dir_from_suffix(suffix: str) → str

static reset_result_files(suffix: str)

save_eval_res(img_fpath, crit_cell_list, err_list)

get_quantiles(cell_key)

get_criticality_score(cell_hist, cell, q, ev_crit_pix=None, dc=None)

for a given histogram and lower and upper bounds, calculate a score which reflects how critical a given histogram is

:param q: quantile container (with attributes q.lower, …)

get_critical_pixel_info(cell_hist, cell, q_curve, dc=None)

:param q_curve: one of the histogram-quantile curves (e.g. q_upper, q_med, q_lower)

abstractmethod find_critical_cells()

abstractmethod find_critical_cells_for_hist_dict_path(hist_dict_path)

abstractmethod find_critical_cells_for_hist_dict(hist_dict, img_fpath, exclude_cell_keys=None, training_data_flag=False)

find_critical_cells_for_img(exclude_cell_keys=None, save_options=None)

preprocess_cell_borders(cell_key, border_value=65)

evaluate_cell(cell_key, dc=None, save_options=None, force_plot=False, recalc_hist=False)

returns 0 for an uncritical cell, 1 for a critical cell Also saves an evaluation plot for every critical cell

ignore_cell_on_experimental_img(cell_key)

save_cell_for_experimental_img(cell_key, cell_hist, q, cc, save_options)

:param cc: criticality_container

save_experimental_img(fprefix='', fsuffix='')

copy_original_image_to_output_folder(fprefix)

get_corrected_cell(cell_key)

save_corrected_cell(cell_key)

false_positive_correction(false_positive_dir)

This iterates over false positives and adapts the affected histograms such that they are not recognized as anomaly anymore.

fp_correct_for_cell(cell_pict_fpath)

False Positive Correction

save_and_plot_critical_cell(img_fpath, hr_row, hr_col, cell_hist, q, cc, save_options)

:param cc: criticality_container :param q: quantile container

_plot_cell_annotation_data_image(cell_data: ipydex.Container)

_plot_cell_analysis_image__paper(cell_data: ipydex.Container)

This version of the method is adapted for the needs of the paper

_plot_cell_analysis_image(cell_data: ipydex.Container)

static highlight_cell(corrected_cell, cc, hard_blend=True, blend_value=120)

This function creates a “cell-shaped” 2d-array (e.g. (105, 25)) where the critical pixels (specified by cc.crit_pix_mask) are highlighted according to hard_blend_flag and blend_value.

create_symlink(existing_fpath, crit_score)

chimcla.stage_2a_bar_selection.bright(txt)

chimcla.stage_2a_bar_selection.bgreen(txt)

chimcla.stage_2a_bar_selection.bred(txt)

chimcla.stage_2a_bar_selection.yellow(txt)