experchange > fortran

Rudi Gaelzer (02-21-19, 05:29 PM)
Using:
Intel(R) Fortran Intel(R) 64 Compiler for applications running on Intel(R) 64, Version 19.0.1.144 Build 20181018

GNU Fortran (GCC) 8.2.1 20181215 (Red Hat 8.2.1-6)
---------------------------------------------------

This is curious: below there are two versions of a simple program which allocates space for an array and passes it to a subroutine, which in turn has to append some new values to it and pass then it back to the caller.

This implementation employs only allocatable arrays and works with both compilers:
---------------------------------------------------
program tes_array_reallocation_IO2
use iso_fortran_env, only: compiler_version
implicit none
integer :: num, i
real, dimension(:), allocatable :: vec

print'(2a)', 'Compiler: ', compiler_version()
write(*, '(a)', advance= 'no')'num= ' ; read(*,*)num
allocate(vec(num))
call svec(vec)
print'(a, *(g0, x))', 'main: vec: ', (vec(i), i= 1, size(vec))

CONTAINS
subroutine svec(vec)
real, dimension(:), allocatable, intent(inout) :: vec
integer :: i
real, dimension(:), allocatable :: v2
call random_number(vec)
print'(a, *(g0, :, x), /)', 'svec: vec: ', (vec(i), i= 1, size(vec))
allocate(v2(2))
call random_number(v2)
print'(a, *(g0, :, x), /)', 'svec: v2: ', (v2(i), i= 1, size(v2))
vec= [vec, v2]
print'(a, *(g0, :, x), /)', 'svec: vec: ', (vec(i), i= 1, size(vec))
return
end subroutine
end program tes_array_reallocation_IO2
---------------------------------------------------

Compiling & running:
---------------------------------------------------
Compiler: Intel(R) Fortran Intel(R) 64 Compiler for applications running on Intel(R) 64, Version 19.0.1.144 Build 20181018
num= 3
svec: vec: .3920868E-06 .2548044E-01 .3525161
svec: v2: .6669145 .9630555
svec: vec: .3920868E-06 .2548044E-01 .3525161 .6669145 .9630555
main: vec: .3920868E-06 .2548044E-01 .3525161 .6669145 .9630555
---------------------------------------------------
Compiler: GCC version 8.2.1 20181215 (Red Hat 8.2.1-6)
num= 3
svec: vec: 0.485422432 0.887927771 0.252131581
svec: v2: 0.295291543E-01 0.657432735
svec: vec: 0.485422432 0.887927771 0.252131581 0.295291543E-01 0.657432735
main: vec: 0.485422432 0.887927771 0.252131581 0.295291543E-01 0.657432735
---------------------------------------------------

This version employs pointers too. It works with ifort, but gfortran produces garbage:
---------------------------------------------------
program tes_array_reallocation_IO
use iso_fortran_env, only: compiler_version
implicit none
integer :: num, i
real, dimension(:), allocatable, target :: vec
real, dimension(:), pointer :: vecp

print'(2a)', 'Compiler: ', compiler_version()
write(*, '(a)', advance= 'no')'num= ' ; read(*,*)num
allocate(vec(num))
vecp => vec
call svec(vecp)
print'(a, *(g0, x))', 'main: vec: ', (vecp(i), i= 1, size(vecp))

CONTAINS
subroutine svec(vec)
real, dimension(:), pointer, intent(inout) :: vec
integer :: i
real, dimension(:), allocatable, target :: v2
call random_number(vec)
print'(a, *(g0, :, x), /)', 'svec: vec: ', (vec(i), i= 1, size(vec))
allocate(v2(2))
call random_number(v2)
print'(a, *(g0, :, x), /)', 'svec: v2: ', (v2(i), i= 1, size(v2))
v2= [vec, v2] ; vec => v2
print'(a, *(g0, :, x), /)', 'svec: vec: ', (vec(i), i= 1, size(vec))
return
end subroutine
end program tes_array_reallocation_IO
---------------------------------------------------

Compiling & running:
---------------------------------------------------
Compiler: Intel(R) Fortran Intel(R) 64 Compiler for applications running on Intel(R) 64, Version 19.0.1.144 Build 20181018
num= 3
svec: vec: .3920868E-06 .2548044E-01 .3525161
svec: v2: .6669145 .9630555
svec: vec: .3920868E-06 .2548044E-01 .3525161 .6669145 .9630555
main: vec: .3920868E-06 .2548044E-01 .3525161 .6669145 .9630555
---------------------------------------------------
Compiler: GCC version 8.2.1 20181215 (Red Hat 8.2.1-6)
num= 3
svec: vec: 0.576124489 0.327901781 0.198874474E-01
svec: v2: 0.111245155 0.903154492
svec: vec: 0.576124489 0.327901781 0.198874474E-01 0.111245155 0.903154492
main: vec: 0.146011130E-18 0.840943870E-09 0.550949667E-13 0.574532370E-43 0.903154492

Could this be a bug with gfortran? I don't think there is memory leak here.
Wolfgang Kilian (02-21-19, 05:58 PM)
On 21.02.2019 16:29, Rudi Gaelzer wrote:
[..]
> svec: vec: 0.576124489 0.327901781 0.198874474E-01 0.111245155 0.903154492
> main: vec: 0.146011130E-18 0.840943870E-09 0.550949667E-13 0.574532370E-43 0.903154492
> Could this be a bug with gfortran? I don't think there is memory leak here.


This looks very much like a programming error. The allocatable array
'v2' is automatically deallocated upon return from the subroutine
'svec'. Since 'v2' happens to be the target of the dummy pointer 'vec'
which is argument-associated with the pointer 'vecp', that pointer
becomes undefined (dangling) after completion of the subroutine.

Using 'vecp' in an expression thus causes undefined behavior. The
actual results indicate that gfortran has overwritten the memory that
was occupied by 'v2', while in the Intel case it was still intact, just
by chance.

-- Wolfgang
CyrMag (02-21-19, 07:11 PM)
On Thursday, February 21, 2019 at 9:58:05 AM UTC-6, Wolfgang Kilian wrote:
[..]
> by chance.
> -- Wolfgang
> --


Wofgang's analysis is right on: NAG 6.2 gives the following run time error:

Runtime Error: rg.f90, line 13: Reference to dangling pointer VECP

-CyrMag
Rudi Gaelzer (02-21-19, 07:23 PM)
On Thursday, February 21, 2019 at 12:58:05 PM UTC-3, Wolfgang Kilian wrote:
> This looks very much like a programming error. The allocatable array
> 'v2' is automatically deallocated upon return from the subroutine
> 'svec'. Since 'v2' happens to be the target of the dummy pointer 'vec'
> which is argument-associated with the pointer 'vecp', that pointer
> becomes undefined (dangling) after completion of the subroutine.
> Using 'vecp' in an expression thus causes undefined behavior. The
> actual results indicate that gfortran has overwritten the memory that
> was occupied by 'v2', while in the Intel case it was still intact, just
> by chance.
> -- Wolfgang


This seems to be the case for this particular program.
Testing the association status of vec (in svec) and vecp (in main) with associated(), both return true, but I guess this behavior is possible if the pointer is in an undefined status.

If I add the attribute 'save' in the declaration of v2:
real, dimension(:), allocatable, target, save :: v2
the vector is not deallocated on output:

Compiler: GCC version 8.2.1 20181215 (Red Hat 8.2.1-6)
num= 3
svec: vec: 0.225852728E-01 0.501595914 0.352034330
svec: v2: 0.986653209 0.250106096
svec: vec: 0.225852728E-01 0.501595914 0.352034330 0.986653209 0.250106096
main: vec: 0.225852728E-01 0.501595914 0.352034330 0.986653209 0.250106096

However, I based the program tes_array_reallocation_IO on an older test case, in which the array is allocated and given values in the subroutine and then returned to the caller:
----------------------------------------
program tes_output_pointer_array2
implicit none
integer :: n, m
real, dimension(:), pointer :: ap, ap2
write(*, '(a)', advance= 'no')'n= ' ; read(*,*)n
!
call out_arr(n, ap, ap2)
!
write(*, '(/, a)')'In main:'
write(*, '(2(a,g0))')('ap(', m, ')= ', ap(m), m= 1, n)
print*, ''
write(*, '(2(a,g0))')('ap2(', m, ')= ', ap2(m), m= 1, n)
CONTAINS
subroutine out_arr(n, ar, ar2)
integer, intent(in) :: n
real, dimension(:), pointer, intent(out) :: ar, ar2
integer :: m
allocate(ar(n))
call random_number(ar)
write(*, '(/, a)')'In out_arr:'
write(*, '(2(a,g0))')('ar(', m, ')= ', ar(m), m= 1, n)
print*, ''
allocate(ar2(n))
call random_number(ar2)
write(*, '(2(a,g0))')('ar2(', m, ')= ', ar2(m), m= 1, n)
return
end subroutine out_arr
end program tes_output_pointer_array2
----------------------------------------

----------------------------------------
Compiler: GCC version 8.2.1 20181215 (Red Hat 8.2.1-6)
n= 4

In out_arr:
ar(1)= 0.544595659
ar(2)= 0.748196602
ar(3)= 0.982226968
ar(4)= 0.703414083E-01
ar2(1)= 0.560840249
ar2(2)= 0.696956098
ar2(3)= 0.491244197E-01
ar2(4)= 0.799976051

In main:
ap(1)= 0.544595659
ap(2)= 0.748196602
ap(3)= 0.982226968
ap(4)= 0.703414083E-01
ap2(1)= 0.560840249
ap2(2)= 0.696956098
ap2(3)= 0.491244197E-01
ap2(4)= 0.799976051
----------------------------------------

In this case, the memory allocated inside the subroutine is not lost and the pointers correctly transfer the memory addresses to the main program. Only if I allocate a temporary targeted array inside the routine without the save attribute that there is memory leak.

Am I correct here? It seems that if space is allocated in the routine for a pointer array, then the memory will not be automatically deallocated whenreturning the pointer to the caller. However, if I allocate an allocatable array (without save), make it a target for a pointer and then pass the pointer back to the caller, then the memory CAN be lost.
Rudi Gaelzer (02-21-19, 07:33 PM)
On Thursday, February 21, 2019 at 2:11:48 PM UTC-3, CyrMag wrote:
> On Thursday, February 21, 2019 at 9:58:05 AM UTC-6, Wolfgang Kilian wrote:
> Wofgang's analysis is right on: NAG 6.2 gives the following run time error:
> Runtime Error: rg.f90, line 13: Reference to dangling pointer VECP
> -CyrMag


Interesting. How does the executable knows that the pointer is dangling? It compares the contents of the memory before and after the transfer?
CyrMag (02-21-19, 07:56 PM)
On Thursday, February 21, 2019 at 11:33:48 AM UTC-6, Rudi Gaelzer wrote:
> On Thursday, February 21, 2019 at 2:11:48 PM UTC-3, CyrMag wrote:
> > Wofgang's analysis is right on: NAG 6.2 gives the following run time error:
> > Runtime Error: rg.f90, line 13: Reference to dangling pointer VECP
> > -CyrMag

> Interesting. How does the executable knows that the pointer is dangling?
> It compares the contents of the memory before and after the transfer?


As is the case with catching errors of any kind at run time, the compiler has to be requested to output extra code to catch the error. The option flags that need to be specified are specific to the compiler used. I used "-C=all -gline".

-- CyrMag
Wolfgang Kilian (02-22-19, 10:45 AM)
On 21.02.2019 18:23, Rudi Gaelzer wrote:
[..]
> ----------------------------------------
> In this case, the memory allocated inside the subroutine is not lost and the pointers correctly transfer the memory addresses to the main program. Only if I allocate a temporary targeted array inside the routine without the save attribute that there is memory leak.
> Am I correct here? It seems that if space is allocated in the routine for a pointer array, then the memory will not be automatically deallocated when returning the pointer to the caller. However, if I allocate an allocatable array (without save), make it a target for a pointer and then pass the pointer back to the caller, then the memory CAN be lost.


Effectively, yes.

Allocating an allocatable array and allocating a pointer array (or
scalar) are fundamentally different concepts. Let me recall:

The memory allocated for an allocated array (say, A) is tied to the
named entity A. If A has the TARGET attribute, you can additionally
have pointers refer to that array. Such pointers have access to the
value but get no control over the allocation status. (However, you can
transfer control to another named *allocatable* array B via MOVE_ALLOC,
which then breaks the connection with A, and you can transfer control
temporarily by argument association.) The rules for allocatable objects
are such that when A goes out of scope, the memory connected to it is
automatically deallocated. There is no way to create a memory leak, but
any pointers referring to such deallocated memory become dangling. If A
has the SAVE attribute, it never goes out of scope. If the named array
that controls the memory retains the TARGET attribute (this is not
trivial!), pointers referring to it are safe.

OTOH, allocating a pointer array B will *first* allocate an anonymous
array in memory and *then* let a pointer B refer to it. In later
statements, you can allow other pointers refer to it, and such pointers
can access both the contents and the allocation of the anonymous target
in memory. If any of those pointers deallocates the target, all other
pointers referring to the target become dangling.
If all pointers currently referring to the target go out of scope, the
target becomes inaccessible but is not deallocated. You got a memory leak.

All this is consistent but nevertheless complicated. My personal rule
is that I never mix named allocatables with pointer-allocated objects.
If possible, I use the former - but if pointer-allocated entities are
already there, I stick exclusively with pointer allocation and prepare
for doing all memory management manually. I found such management
straightforward as long as I remember that any allocated object is
anonymous, automatically has the TARGET attribute, and exists
independently of pointers referring to it.

-- Wolfgang
Rudi Gaelzer (02-22-19, 03:28 PM)
On Friday, February 22, 2019 at 5:45:53 AM UTC-3, Wolfgang Kilian wrote:
[..]
> straightforward as long as I remember that any allocated object is
> anonymous, automatically has the TARGET attribute, and exists
> independently of pointers referring to it.


This is also my conclusion after doing some digging around. I have to confess that I still don't have a firm grasp on how pointers in fortran work...the situation is a bit worse because the manner on which fortran pointers are described by the literature is somewhat "opaque". I miss a good description of the behavior of pointers in the language.

With C, OTOH, the confusion created by the distinction between the
value of the pointer (the memory address) and the value of the data stored in that address is so great that it has forced the publication of some books dealing with the issue. The book "Understanding and Using C Pointers" byRichard Reese is my reference.
I've just found there an example of dangling pointers that mirrors the problem I reported. In pages 64-66 the author shows why you can get in troublewhen you return the address of a local array that becomes undefined when the function goes out of scope. OTOH, when you allocate space with malloc (in C), the heap manager will reserve the corresponding space somewhere outside of the function's stack. In this way, the reference is not lost when control is returned to the caller.

It seems to me that something similar is ocurring in my examples. The solution I found with the save attribute works because the reference to the allocated space is never lost, similar to what happens in C when you declare the local array as static.
Rudi Gaelzer (02-22-19, 04:54 PM)
On Friday, February 22, 2019 at 11:08:55 AM UTC-3, edmondo.g...@gmail.com wrote:
> Well, Fortran pointers are more like weak references than C pointers.
> They are variables that doesn't own the memory they reference.
> So, if, for other reasons, that memory is not anymore available they remain dangling as nobody has told them that that memory is not available anymore.
> It's the programmer that should take care of that.
> That means that you should use allocatable variables whenever you can if you need dynamic memory.


"Weak reference" as in the sense as described here:?

I'm not a CS, so I think this is the first time I'm hearing this...
I'm sure everyone in this forum at least once heard in the grapevine that fortran pointers are more like "aliases" than true pointers. But what does it mean?
Since a C pointer is a variable by itself, it holds a fixed space in memory regardless of the type. It can be 32 or 64 bits depending on the architecture, and pointers to functions can have different lengths.
An "alias" assumes that the pointer is something like a link, which also takes some memory space. However, as Steve Lionel points out here:

in Fortran, a pointer can have additional information such as array bounds and strides, which implies that the length of the pointer itself is variable.

I think it would be nice if someone sometime would take a time to write a detailed description (for non-CS people) of fortran pointers and the differences to other languages, mostly C/C++ and python.
FortranFan (02-22-19, 05:13 PM)
On Friday, February 22, 2019 at 9:54:50 AM UTC-5, Rudi Gaelzer wrote:

> ..
> I think it would be nice if someone sometime would take a time to write adetailed description (for non-CS people) of fortran pointers and the differences to other languages, mostly C/C++ and python.


@Rudi Gaelzer,

Chances are the terms like 'weak references', 'aliases', etc. applied toward the POINTER attribute (note it's need an ATTRIBUTE of an object per Fortran standard) mean essentially the same thing. You can refer to 'Modern Fortran Explained' by Metcalf et al. and 'Modern Fortran: Style and Usage' by Clerman and Spector to read pretty much all there is to know about this feature in the language.

Or during the intervening time until a 'treatise' on the POINTER attribute of variables in Fortran appears, if you can convey what exactly you are trying to achieve in your codes and what makes you consider using the POINTER attribute over the ALLOCATABLE one, readers here might be able to provide you with most of the tips and caveats you need to consider.
Wolfgang Kilian (02-22-19, 05:59 PM)
On 22.02.2019 15:54, Rudi Gaelzer wrote:
> On Friday, February 22, 2019 at 11:08:55 AM UTC-3, edmondo.g...@gmail.com wrote:
> "Weak reference" as in the sense as described here:?
>
> I'm not a CS, so I think this is the first time I'm hearing this...
> I'm sure everyone in this forum at least once heard in the grapevine that fortran pointers are more like "aliases" than true pointers. But what does it mean?
> Since a C pointer is a variable by itself, it holds a fixed space in memory regardless of the type. It can be 32 or 64 bits depending on the architecture, and pointers to functions can have different lengths.
> An "alias" assumes that the pointer is something like a link, which also takes some memory space. However, as Steve Lionel points out here:
>
> in Fortran, a pointer can have additional information such as array bounds and strides, which implies that the length of the pointer itself is variable.
> I think it would be nice if someone sometime would take a time to write a detailed description (for non-CS people) of fortran pointers and the differences to other languages, mostly C/C++ and python.


It might be easier to completely forget about C/C++ pointers when
considering pointers in Fortran. Listing differences can only lead to
more confusion.

A more appropriate analog to a Fortran pointer is a symbolic link as a
concept of Unix-like file systems. I find it very natural to work with
symbolic links.

Complications arise when you allocate anonymous targets (in Fortran).
In the Unix analog, this would be an ordinary file which is only
accessible by symbolic links. If you can avoid this situation, you
should try to avoid it.

-- Wolfgang
Ev. Drikos (02-22-19, 10:25 PM)
On 22/02/2019 4:54 PM, Rudi Gaelzer wrote:
> ...
> "Weak reference" as in the sense as described here:?
> ...
> in Fortran, a pointer can have additional information such as array bounds and strides...


Here is a snapshot of your 2nd program as reported by LLDB, immediately
after the pointer assignment "vecp => vec".

With GNU Fortran, the allocatable variable "vec" and the Fortran pointer
"vecp" are represented by two distinct C structures.

Unluckily, the Fortran pointer "vecp" does have access only to the
actual data of the Fortran variable "vec", since the C pointer "data" in
both structures have the same value. They are two different C pointers
with the same value, which are otherwise unrelated.

vec array1_real(kind=4)
data void * 0x102100ec0 0x0000000102100ec0
offset long -1
dtype long 281
dim descriptor_dimension []

vecp array1_real(kind=4)
data void * 0x102100ec0 0x0000000102100ec0
offset long -1
dtype long 281
dim descriptor_dimension []
Similar Threads