Skip to content

Add numpy v2 support

Thomas Vincent requested to merge tvincent/txs:numpy2 into master

This MR makes small changes for the code to run with numpy v2:

This MR does NOT address changes to numpy data type promotion, so the code should run fine but results might be stored with a different precision (eventually leading to overflows...).

For more details, see numpy 2 migration guide.

I've tested this in an environment with:

All those packages should be released with numpy2 support before the effective release of numpy2.

I tested it through the GUI and run the tests with pytest. Some tests fails (they also fail with numpy v1):

  • Due to missing files in the repo: mask.edf -which I took from an older version- and .log file
  • Not getting the same sum when reading EDF and bliss files
Test errors 
_____________________________________________________________ test_edf_datared _____________________________________________________________

edf_data_path = '/users/tvincent/src/txs/txs/tests/sample_data/edf/'

    def test_edf_datared(edf_data_path):
        azav = integrate1d_dataset(edf_data_path, **_TEST_KEYWORDS)
>       red = datared(azav, 'auto', norm=(2.1, 2.2), red_chi2_max=5)

txs/tests/test_datared.py:25: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
txs/datared.py:74: in datared
    log = read_id09_log_file(data['folder'])
txs/datasets.py:344: in read_id09_log_file
    return read_id09_log_file(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

fname = '/users/tvincent/src/txs/txs/tests/sample_data/edf', skip = None, format = 'waxscollect', out_format = 'datastorage'
get_info = False

    def read_id09_log_file(fname, skip=None, format='auto',
                           out_format='datastorage', get_info=False):
        """Read ID09 log file associated to a tr-waxs dataset.
    
        Parameters
        ----------
        fname : str
            Name of log file or folder. If 'fname' is a folder, the file name
            will be guessed automatically.
        skip : int or tuple or None, optional
            First/last lines to be skipped. Default is None.
        format : str {'waxscollect', 'bliss', 'auto'}, optional
            Log file format. Default is 'waxscollect'.
        out_format : str, optional
            Can be 'ndarray', 'dict' or 'datastorage'. Default is 'datastorage'.
        get_info : boot, optional
            If True, extra info (parameters value) is also returned.
            Default is False.
    
        Returns
        -------
        data : ndarray or dict or DataStorage
            Retrieved data from id09 log file.
            Use 'out_format' to choose between np.genfromtxt() ndarray or dict.
        info : dict
            Retrieved info from id09 log file comment section. Returned only
            if 'get_info' is True.
    
        """
    
        if format == 'waxscollect':
            if os.path.isdir(fname):
                fnames = get_fnames("*.log", folder=fname)
    
                if len(fnames) == 0:
                    full_path = os.path.join(os.getcwd(), fname)
>                   raise Exception("No *.log file found in '%s'." % full_path)
E                   Exception: No *.log file found in '/users/tvincent/src/txs/txs/tests/sample_data/edf'.

txs/datasets.py:247: Exception
----------------------------------------------------------- Captured stdout call -----------------------------------------------------------

Loading results from previously stored analysis: /users/tvincent/src/txs/txs/tests/sample_data/edf/id09_azav.h5
Loading time: 0.032 s

Reading ID09 log file...
___________________________________________________________ test_load_edf_images ___________________________________________________________

edf_data_path = '/users/tvincent/src/txs/txs/tests/sample_data/edf/'

    def test_load_edf_images(edf_data_path):
        imgs, fnames = load_images(
            edf_data_path,
            extension='edf',
            exclude=["run01_0004.edf"],
            return_fnames=True
        )
>       assert np.sum(imgs) == 1082970310
E       assert np.uint64(18052713070) == 1082970310
E        +  where np.uint64(18052713070) = <function sum at 0x7f4fb43c28f0>(<txs.datasets.ImageIteratorEDF object at 0x7f4f3c3b11f0>)
E        +    where <function sum at 0x7f4fb43c28f0> = np.sum

txs/tests/test_datasets.py:19: AssertionError
__________________________________________________________ test_bliss_scan_slice ___________________________________________________________

bliss_scan = <txs.datasets.ImageIteratorHDF5 object at 0x7f4f3c3b0e30>

    def test_bliss_scan_slice(bliss_scan):
>       assert bliss_scan[:10].sum() == 1478169543
E       assert np.uint64(10068104135) == 1478169543
E        +  where np.uint64(10068104135) = <built-in method sum of numpy.ndarray object at 0x7f4f3c2d3150>()
E        +    where <built-in method sum of numpy.ndarray object at 0x7f4f3c2d3150> = array([[[ 0,  0,  0, ..., 12, 13, 12],\n        [ 0, 34, 36, ..., 28, 32, 13],\n        [ 0, 36, 34, ..., 35, 36,  0],\n ...36, 34, ..., 32, 34, 12],\n        [12, 33, 33, ..., 24, 34, 12],\n        [12, 12,  0, ...,  0,  0,  0]]], dtype=uint16).sum

txs/tests/test_datasets.py:56: AssertionError

Merge request reports