Namespaces and generic functions in R

Question

this question is somewhat a follow up on this question. Consider the following example

set.seed(1)
x <- cumsum(rnorm(10))
y <- stats::arima(x, order = c(1, 0, 0))
length(stats::fitted(y))
[1] 0

So far so good: zero is returned because R does not now how to use stats::fitted on an object of class Arima.

Next in my code, I need one function from the forecast package. I do not attach the package, I just load it using the ::notation. In my code below I will load it directly using requireNamespace.

requireNamespace("forecast", quietly = TRUE)
length(stats::fitted(y))
[1] 10

And suddenly the same command returns a different result. I understand why this happens (and I hope I am saying it correctly): by loading the forecastpackage a new method for the generic function fitted (namely fitted.Arima) is loaded into the namespace which results in a different outcome.

For me this behavior is quite annoying: is there any way to choose one specific method for fitted? I read this chapter but did not figure out how to circumvent this problem.

I also tried to unload the forecast package from namespace, but no success:

unloadNamespace("forecast")
length(stats::fitted(y))
[1] 10

It seems that once I load the package I cannot use the old method of fitted. I am wondering how to handle these situations.

EDIT

As pointed out in the comments after unloadNamespace("forecast") I get that

isNamespaceLoaded("forecast")
[1] FALSE

But methods fitted still includes fitted.Arima.

Answer 1

I found this thread from R devel. Brian Ripley (of R Core) says:

Unloading a namespace does not unregister its methods (and registration has no stack, so there is no way R knows what was there before).

The thread then notes that ?unloadNamespace points you at ?detach:

See the comments in the help for detach about some issues with unloading and reloading name spaces.

which does eventually say the following (emphasis mine)

If a package has a namespace, detaching it does not by default unload the namespace (and may not even with unload = TRUE), and detaching will not in general unload any dynamically loaded compiled code (DLLs). Further, registered S3 methods from the namespace will not be removed.

My understanding is therefore that while loading a namespace (such as by using ::) registers S3 methods, these methods are never linked to the namespace that they were loaded from and so unloading the name space cannot also unregister the methods. The only way to clear them from methods() would be to restart R.

As RolandASc noted, you can choose to call the default method by using stats:::fitted.default if you want to avoid dispatch to fitted.Arima.

Answer 2

@CalumYou is exactly right in pointing out that unloading a namespace will not remove S3 methods registered for an S3 generic defined in another package. Here, in case you are interested, is a more detailed look at how and why that is the case.

When the forecast package is loaded, all of the methods that it defines are "registered" in data bases in a variety of different namespaces. The rule R follows is that a method gets registered in the namespace of the package that defines its S3 generic. Since the fitted() generic is defined in stats, that's where the new methods defined by forecast get registered, in an environment called .__S3MethodsTable__.. Detaching or unloading forecast leaves the stats package untouched (probably an overall wise design decision, if you think about it), with the unfortunate consequence that the fitted.Arima method (along with many others) remain registered in its .__S3MethodsTable__.

To see that this is so, have a look at the following:

isNamespaceLoaded("forecast")
## [1] FALSE
ls(stats:::.__S3MethodsTable__., pattern = "fitted")
## [1] "fitted.default"       "fitted.isoreg"        "fitted.kmeans"       
## [4] "fitted.nls"           "fitted.smooth.spline"

## Loading the forecast namespace registers new 'fitted' methods ...
requireNamespace("forecast", quietly = TRUE)
isNamespaceLoaded("forecast")
## [1] TRUE
ls(stats:::.__S3MethodsTable__., pattern = "fitted")
##  [1] "fitted.ar"              "fitted.Arima"           "fitted.arma"           
##  [4] "fitted.bats"            "fitted.default"         "fitted.ets"            
##  [7] "fitted.fracdiff"        "fitted.garch"           "fitted.gls"            
## [10] "fitted.glsStruct"       "fitted.gnls"            "fitted.gnlsStruct"     
## [13] "fitted.isoreg"          "fitted.kmeans"          "fitted.lagwalk"        
## [16] "fitted.lme"             "fitted.lmeStruct"       "fitted.lmList"         
## [19] "fitted.modelAR"         "fitted.nlmeStruct"      "fitted.nls"            
## [22] "fitted.nnetar"          "fitted.quantmod"        "fitted.smooth.spline"  
## [25] "fitted.tbats"           "fitted.tslm"            "fitted.values.quantmod"

## ... which are left behind even when the forecast namespace is unloaded
unloadNamespace("forecast")
isNamespaceLoaded("forecast")
## [1] FALSE
ls(stats:::.__S3MethodsTable__., pattern = "fitted")
##  [1] "fitted.ar"              "fitted.Arima"           "fitted.arma"           
##  [4] "fitted.bats"            "fitted.default"         "fitted.ets"            
##  [7] "fitted.fracdiff"        "fitted.garch"           "fitted.gls"            
## [10] "fitted.glsStruct"       "fitted.gnls"            "fitted.gnlsStruct"     
## [13] "fitted.isoreg"          "fitted.kmeans"          "fitted.lagwalk"        
## [16] "fitted.lme"             "fitted.lmeStruct"       "fitted.lmList"         
## [19] "fitted.modelAR"         "fitted.nlmeStruct"      "fitted.nls"            
## [22] "fitted.nnetar"          "fitted.quantmod"        "fitted.smooth.spline"  
## [25] "fitted.tbats"           "fitted.tslm"            "fitted.values.quantmod"

(For a related question and answer, see here.)