The State of Fortran Generics

I just returned from the joint WG5/J3 meeting (the international and US committees in charge of producing the next revision to the Fortran standard). The Generics subgroup, of which I am a contributing member, had a very successful showing.

The committee discussed and passed 4 “Specification” papers regarding the template feature slated for inclusion in the 202Y revision of the standard. The combination of the papers provide a complete description of the expected semantics of the feature. In this post I will try to summarize and demonstrate with some examples, what this feature will enable, and likely look like.

NOTE: The exact syntax has not been decided. There are many keywords and syntax elements that are still in debate, but the general structure should not change much. It’s entirely likely that the examples shown below will not work unmodified when the standard is finally published.

The four papers passed, and that I will try to summarize in order, define the semantics for

  • A new template construct, including where it may appear, and what may appear within it
  • The scoping rules for templates and their instantiations
  • A new instantiate statement
  • A new restriction block and requires statement

Template Construct

The first thing of note is that a new construct will be available. This will be used to define a template, with specific things being “parameterized”. The things allowed to be template parameters are:

  • types
  • procedures
  • integer, logical or character constants (Note that characters must be assumed length, and arrays must be assumed size or assumed rank)

A template may appear in the specification section of a

  • module
  • submodule
  • template

A template can then contain any valid Fortran that could be found within a module. I.e. it can define new types, variables, constants and procedures following a contains statement. There is one caveat. All operations and procedure invocations within a template must have explicit interfaces, and for the purposes of checking those interfaces, all deferred types (types that are template parameters) are treated as completely unique. This has the implication that entities of deferred type; can only be assigned to entities declared to be of the same deferred type, can only be passed as actual arguments to procedures who’s corresponding dummy argument is declared to be of the same deferred type. The consequences of such a constraint is that it can be determined a priori that a template will be valid for all actual parameters. The following, somewhat contrived, example illustrates the intended behavior.

type :: u
  ...
end type
...
template tmpl(T, C, S)
  type, deferred :: T
  character(len=*), constant :: C
  interface
    subroutine S(x, y)
      type(T), intent(inout) :: x, y
    end subroutine
  end interface
contains
  function f(x, i)
    type(T), intent(in) :: x
    type(u), intent(in) :: i
    type(T) :: f
    ...
  end function
  subroutine foo(x, u1)
    real, intent(inout) :: x
    type(u), intent(in) :: u1
    type(T) :: t1, t2

    call S(t1, t2) ! Valid
    call S(t1, x)  ! Invalid; x not declared to be of type T
    t1 = f(t2, u1) ! Valid
    x = f(t1, u1) ! Invalid; cannot assign T to real
    t1 = t2 + t2 ! Invalid; + not defined for T + T
  end subroutine
end template

Scoping Rules

The scoping rules for templates are relatively straightforward at this point. A template has host association (i.e. it has access to any entities available in the scope in which it is defined). An instantiation of a template brings into scope only those entities within the template that are public. Thus it will likely be best practice for templates, as many consider it to be for modules, for a template to begin

template tmpl(...)
  private
  public :: ...
  ...
end template

as well as for instantiations to make use of the only clause. I.e.

instantiate tmpl(...), only: ...

Instantiation

A new instantiate statement provides actual parameters for a template, and makes the template entities available. It also provides a rename mechanism to alleviate any potential name conflicts, and an only clause to limit those entities actually brought into scope. There is some complication involved in the underlying mechanism for this however. An instantiate statement is said to refer to an instantiation, and that instantiate statements with identical actual parameters are said to refer to the same instantiation.

The main benefit to this is that types declared within a template and then instantiated in multiple places are the same type where the rules of Fortran might otherwise consider them to be different types. It also means that procedures with save variables, while not advised, will behave as expected when used in separate places (i.e. the “separately instantiated procedures” will refer to the same saved variable). It will be possible to override this behavior, so that an instantiate statement can produce an entirely separate and unique instance. An example to illustrate is shown below.

template wrapper_tmpl(T)
  type, deferred :: T
  type :: wrapper
    type(T) :: wrapped
  end type
end template

instantiate wrapper_tmpl(real), real_w => wrapper
instantiate wrapper_tmpl(real), other_w => wrapper
instantiate, unique :: wrapper_tmpl(real), w1 => wrapper
instantiate, unique :: wrapper_tmpl(real), w2 => wrapper

! Types real_w and other_w are the same type.
! Type w1 is a different type than all of real_w, other_w and w2
! Type w2 is a different type than all of real_w, other_w and w1

For well designed templates and libraries, users shouldn’t have to think about these complexities most of the time. For compiler writers however, this complexity could be tricky. An instantiate statement may need to refer an instance produced by a previously created instantiate statement, in which case it needs to somehow find it, or it may need to create one in such a way that other instantiate statements will be able to find it.

Restriction and Require

Because certain combinations of template parameters are likely to be common, and have meaningful names, another new block and statement have been added to facilitate the naming and reuse of certain template parameter declarations. A restriction block is a new construct, with a name and template parameters, that can contain declarations of template parameters. A requires statement can then appear in a template or restriction block to “include” those declarations. An illustrative example is shown below.

restriction binary_op(T, U, V, binop)
  type, deferred :: T, U, V
  interface
    function binop(x, y) result(z)
      type(T), intent(in) :: x
      type(U), intent(in) :: y
      type(V) :: z
    end function
  end interface
end restriction
...
template tmpl(T, binop)
  requires binary_op(T, T, real, binop)
contains
  function path_length(steps)
    type(T), intent(in) :: steps(:)
    real :: path_length

    integer :: i

    path_length = 0
    do i = 1, size(steps)-1
      path_length = path_length + binop(steps(i), steps(i+1))
    end do
  end function
end template

Conclusion

The semantics of the new template feature have now been established. The committee still has work to do on finalizing the syntax and then making appropriate edits to the standard, but the basic structure and behavior is now clear.

Acknowledgements

I need to acknowledge all those who helped in this effort. There are too many to name them all, but in particular, Tom Clune (of NASA) has done a tremendous job organizing the subgroup and representing the ideas to the committee. The committee has also been very understanding and provided us great constructive feedback.

13 thoughts on “The State of Fortran Generics

  1. I’m not 100% versed in OOP. Is the template construct a better way to implement one algorithm for various data types? For example, is this construct the next step after using generic interfaces to cover otherwise identical procedures, e.g. real and integer math? That would be excellent

    Like

  2. This looks completely alien as compared to what other languages offer.

    I am not sure whether I am missing something, but in a modern design, generics should be bounded by traits/interfaces, and the latter should support seamlessly both compile-time and run-time polymorphism (as it is e.g. the case in Rust, or Carbon, see below).

    – Where are those traits/interfaces here? They should be absolutely central to a modern design.
    – How would one implement such a trait for a derived type, using the above syntax?
    – Does the approach taken by the Fortran committees support merely compile-time or also run-time polymorphism?
    – If it supports both, would it be seamlessly possible to switch between the two (as in Rust) by using the same trait?

    See also
    https://github.com/carbon-language/carbon-lang/blob/trunk/docs/design/generics/overview.md
    for everything I’ve mentioned above.

    Like

    1. I’m somewhat familiar with traits/type-classes/interfaces from some other languages. We explored this avenue for the design of generics in Fortran, but found it didn’t solve all our use cases while interacting well with other existing features of the language. That said, the restriction block and some well thought out conventions are intended to resemble something like traits (if you squint a little bit).

      For now, templates are really only for compile-time polymorphism. We’d be open to suggestions for future improvements that would enable more run-time polymorphism, but our current use cases didn’t seem to require it.

      I’d say if we were designing a new language from scratch, traits are a great design (but not the only viable one). Trying to retroactively fit them into a language with such a “rich history” as Fortran is … much harder.

      Like

      1. This is factually incorrect. The language already has (unnamed) abstract interfaces, which could be extended to traits by simply providing named versions of them. This would make it possible to easily support both run-time and compile-time polymorphism with the *same* design (as in Swift, in which
        protocols interact *perfectly* with all the other features of the language, like e.g. class inheritance, so the
        above statement is factually wrong).

        To forgo what I’ve just mentioned, means to provide a patchwork of features instead of a well-thought out unified solution. But I’d say I am not surprised. This only confirms that there’s no vision in the committee for the future of Fortran. There’s a reason why “design by committee” is frowned upon by those who understand language design. It littered Fortran with such abominations as “unlimited polymorphic objects”, implementation inheritance, procedure pointers, the “select type” statement, parametrized derived types, and other anti-features.

        Things that should never have been adopted from other languages were repeatedly adopted by the committees, while good designs were/are not just foregone, but replaced by inferior home-brew ideas.

        I am also not sure who is meant with “… our use-cases”. I guess it means the committees’. The users certainly think differently (I am sure, I do).

        If the above design is indeed adopted for generics, it will be the death sentence for the language.

        Like

      2. I’m sorry you feel that way, but subgroup has worked hard, done its best, and I’m proud of the design we’ve come up with. I’m not sure I see clearly the design you have in mind. Would you be willing to write up a sufficiently comprehensive specification for your idea that the committee could evaluate it versus the current design? It would still be possible to change our minds.

        Sure, language design by committee has its drawbacks, but it’s where we are with Fortran. Might I ask why you feel compelled to provide such criticism publicly after-the-fact, but not sufficiently invested to actually collaborate with the committee on this work?

        Also, the committee, and specifically the generics subgroup, did solicit input for use cases from anyone willing to provide them. So if there are users who think differently and have use-cases not satisfied by this design, it’s unfortunate they were unwilling to provide such insights to the committee. We’d still be open to entertaining new use cases if such persons would be willing to provide them.

        Like

  3. Your assumptions are again incorrect.

    I have submitted a 17 page long proposal (including use cases, and even example codes in Rust and Java) to J3’s Github page on how the run-time polymorphism part ought to be handled using abstract interfaces/traits (in the framework of the committee’s greater generics effort), some two years ago (I am sure you are aware of this proposal). With the understanding that the committee would incorporate the ideas of this proposal, and extend them to include also traits-based compile-time polymorphism (a la Rust, Swift, etc).

    Now I see that none of what I suggested has been taken into account. No abstract interfaces/traits, no run-time polymorphism based on multiple interface inheritance, no nothing. I’ve wasted valuable time on this proposal. Time that I had to steal from important projects, only to be completely ignored. So to accuse me of not being willing to work with the committee on these things is strange, to say the least.

    If the committee truly believes that Fortran can do without these things, then Fortran users will vote with their feet. I certainly will, and I am sure others will, too.

    Like

      1. I went back and reread the PR and proposal. I apologize for not remembering it. My only excuse is that a lot happened over the 2+ years since I saw it. I object to the assertion that the committee didn’t consider it. Clearly many on the committee engaged with it at the time. But a lot of additional work has to be done to get something into the standard.

        We followed what seemed like a pretty reasonable process. Identify specific use cases, i.e. problems we’d like it to be easier to solve in Fortran. From there, derive requirements that would enable solutions to those use cases. From the requirements we designed a specification of the feature. I don’t recall off the top of my head whether we explicitly considered this design once we got into the details of the specification or even if we were thinking about it by the time we got to requirements, but it seems it wasn’t the obvious answer by that point.

        I apologize if we just missed it/forgot about it, but unfortunately writing one paper, throwing it over the wall and expecting the committee to do the rest of the work for you just isn’t sufficient to get exactly what you want. It takes active participation all the way through the process to make sure your ideas are represented.

        I can’t promise we would completely redo all the work over the last year or so to write the requirements and specs, but if somebody were so willing and a compelling case could be made we might consider it. What it would take at this point would be to demonstrate that this addresses our original use cases in a more convenient way. I can send you that document if you wish.

        Like

  4. I didn’t intend to make an additional comment, but I feel compelled to, for Fortran’s sake.

    The committee’s proposed design completely misses the main *point* of generic programming (of which OO programming/run-time polymorphism is a special case) namely to make a program’s source code depend on *abstractions* instead of concretions. The abstractions are the interfaces/traits that I mentioned above, which this proposal ignores. Well-written generic code should have merely such interfaces/traits as dependencies, and should not depend on *anything* else.

    In case of the above design, every defined template is a concretion. Making use of such a template from program units other than the unit/scope in which the template was defined, would introduce a dependency upon a concretion into the importing program unit. This would reduce ad absurdum the entire raison d’etre for generics in the language.

    I’d therefore like to ask you, to please take a look at the Carbon language Github page link that I provided above.
    It provides a detailed and well written account that explains why the design they’ve chosen, and which would be a great fit for Fortran, is to be preferred over anything template related.

    Chandler Carruth, the main author of Carbon is actually a well-know C++ programmer, and he and other C++ people have learned a great deal from all the mistakes of C++. It is telling that they chose a trait/interface based design for Carbon.

    For Fortran’s sake, please reconsider your approach!

    Like

    1. I disagree that this design forces code to depend on concretions. The restriction blocks are the interfaces/traits, and a template can depend entirely on the abstractions. A template could even be used from within another template and still depend solely on abstractions. It is true that at some level an instantiate must provide concrete types and procedures, but that’s true of generics in all forms in all languages.

      The biggest differences here are that restrictions are not defined in terms of type-bound procedures, restrictions need not be (and at this point can’t be) satisfied by type-bound procedures, and derived types need not declare their ability to satisfy some restriction, but this seems like added flexibility, not a loss of functionality. Admittedly, this design can get verbose, but I don’t see where we’ve lost some fundamental capability.

      Again, if you have a use case in mind that you think this design cannot accommodate, I’m open to exploring it.

      Like

      1. If “restrictions” are your interfaces/traits, as you say, then I cannot see why one would want to have an additional such feature, that is different from the one the language already has, namely abstract interfaces.

        This goes against orthogonality in the language. It only complicates the language, for – in my opinion – very questionable, if any, gain. In this case, present OO code that uses abstract interfaces and dynamic dispatch couldn’t be easily switched over to use static dispatch.

        Anyway, I guess our views on this are too different to be reconciled. So let’s just agree that we disagree.

        Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s