[{"data":1,"prerenderedAt":686},["ShallowReactive",2],{"post-compile-unit-neq-namespace":3},{"id":4,"title":5,"aliases":6,"author":6,"authors":7,"body":9,"cover":6,"date":677,"description":678,"draft":6,"excerpt":6,"extension":679,"extra":6,"meta":680,"navigation":681,"pageKind":6,"path":682,"seo":683,"stem":684,"taxonomy":6,"unlisted":6,"__hash__":685},"posts\u002Fposts\u002Fcompile-unit-neq-namespace\u002Findex.md","Please don't make your compile unit the same size as your namespace.",null,[8],"Rynco Maekawa",{"type":10,"value":11,"toc":664},"minimark",[12,16,56,59,81,84,87,109,112,118,132,138,151,156,164,169,172,179,193,202,205,223,228,233,236,239,242,245,250,253,257,261,264,271,281,284,301,310,315,328,338,345,352,365,368,373,379,382,385,390,395,402,408,422,431,436,446,449,453,458,461,468,475,478,487,493,499,502,506,509,516,521,525,528,535,538,541,546,551,558,564,569,576,579,590,593,603,606,617,632,636,644,653,656,661],[13,14,15],"p",{},"Before we start,\nallow me to specify the terminology used in this article.\nTerms like \"package\" and \"module\" tend to differ from language to language,\nso it's good to settle on one interpretation so we avoid mimatches.",[17,18,19,32,47],"ul",{},[20,21,22,23,27,28,31],"li",{},"A ",[24,25,26],"strong",{},"package"," is the unit of code that usually gets distributed atomically.\nExamples are NPM and Python packages, Java JARs, Rust crates, Go modules.",[29,30],"br",{},"When mentioning Go packages (namespaces), it is always next to \"Go.\"",[20,33,22,34,37,38,42,43,46],{},[24,35,36],{},"compile unit"," is the collection of code that gets compiled\nin one call to the compiler.\nThink whatever files you feed into ",[39,40,41],"code",{},"gcc"," or ",[39,44,45],{},"javac",".\nUsually, cyclic dependency can only exist within one compilation unit,\nunless the language contains features like forward declaration.",[20,48,22,49,52,53,55],{},[24,50,51],{},"namespace"," is an organization of code symbols that is logically grouped together.\nReferencing to symbols (functions, vars, types) within the same namespace\nis usually easier than referencing those outside.\nSymbols can have the same name in different namespaces,\nbut usually can't within the same namespace (except overloading).",[29,54],{},"Examples are C++ and C# namespaces, Java and Go packages, Rust and OCaml modules.\nLow-level languages like C don't have namespaces.",[57,58],"hr",{},[60,61,64,65,68,69],"chat",{":avatar":62,":flip":63},"rynco.jpg","true","  ",[13,66,67],{},"\n    Yeah I'm not a Go expert.\n  ","\n  ",[13,70,71,72,76,77,80],{},"\n    I have ",[73,74,75],"em",{},"read"," a couple of Go code before for researching and debugging,\n    but almost never ",[73,78,79],{},"written"," them.\n    Take my statements with a grain of salt if you like.\n  ",[13,82,83],{},"I have been researching about module systems between programming languages recently,\nbecause I'm troubled constantly by one specific design decision.",[13,85,86],{},"You see, compiled programming languages with namespaces\noften fall into one of two categories:",[17,88,89,99],{},[20,90,91,92,95,96,98],{},"Some ",[24,93,94],{},"have compile units spanning multiple namespaces",".\nAn entire package is usually compiled with only one compiler invocation.",[29,97],{},"Examples of this are: C#, Java, Dart, Rust, etc.",[20,100,101,102,105,106,108],{},"Some others ",[24,103,104],{},"have namespaces spanning multiple compile units",",\nusually using one compiler invocation per file.\nSome of them can also declare smaller namespaces within them.",[29,107],{},"Examples of this are: C++, OCaml (with wrapped modules).",[13,110,111],{},"Either way (or for some language both ways),\nthe size of a namespace is usually different from the size of a compile unit.\nThe compiler leaves you some room to logically groups parts of your code into smaller units,\nso you may maintain cleanness while writing namespaces that reference each other.",[13,113,114,117],{},[24,115,116],{},"But there's one language that's a big exception of that -- Go",".",[13,119,120,121,131],{},"In Go, ",[24,122,123,124,127,128],{},"a single folder within the source code repository simultaneously respresent\n",[73,125,126],{},"one compile unit"," and ",[73,129,130],{},"one namespace",", which Go calls a \"package\".\nGo packages cannot have dependency cycles,\nand neither does it have any smaller internal structures.\nAll symbols declared within it must reside in one flat namespace,\nwhich spans all source code files within it.",[13,133,134,137],{},[24,135,136],{},"But why is Go so special?","\nCan we study Go's implementation and use it in our own ones?",[60,139,141],{":avatar":140},"nene.jpg",[13,142,143,144,147,148],{},"Isn't it because Go's philosophy is to simplify the ",[73,145,146],{},"compiler"," at every turn,\nwhile happily forcing complexity onto the ",[73,149,150],{},"users?",[60,152,153],{},[13,154,155],{},"QED. End of article.",[60,157,158],{":avatar":62,":flip":63},[13,159,160,163],{},[73,161,162],{},"(chuckles)"," Can't you just shut up and let me explain first?",[165,166,168],"h2",{"id":167},"when-gos-solution-worked","When Go's solution worked",[13,170,171],{},"The structure of Go's package should look familiar to many C and C++ programmers.\nSimply speaking,\nit's the common C\u002FC++ project structure,\nsans header files and forward declaration.",[13,173,174,175,178],{},"In fact, I would suspect that such design directly comes from C projects --\nthe three main designers of it are all famous senior C\u002FC++ engineers,\nwho have worked large, ",[73,176,177],{},"very"," large C and C++ projects.",[13,180,181,182],{},"Needless to say, Go's package system works.\n",[24,183,184,185,188,189,192],{},"But it works not because it's ",[73,186,187],{},"good",".\nRather, it's because it is ",[73,190,191],{},"aligned and deeply coupled"," with the other design aspects of Go.",[60,194,195],{":avatar":140},[13,196,197,198,201],{},"Well at least it's better than writing ",[39,199,200],{},"CMakeLists.txt","s I guess?",[13,203,204],{},"And I want to put my argument upfront.\nGo's package system works now because the following 4 features of Go\nforms a reasoning cycle that self-supports.\nWhen some of them change,\nthe rationale that makes Go packages what they are now no longer holds rigidly.",[206,207,208,211,214,220],"ol",{},[20,209,210],{},"A namespace contains every file in a folder, nothing more or less.",[20,212,213],{},"Go's interfaces are structural, decoupling it from implementation.",[20,215,216,219],{},[39,217,218],{},"go generate"," generates new files instead of macros modifying existing files.",[20,221,222],{},"Each file gets to specify its list of imported namespaces.",[224,225,227],"h3",{"id":226},"namespaces","Namespaces",[229,230,231],"blockquote",{},[13,232,210],{},[13,234,235],{},"It's direct derivation of classic C projects' structures.\nA somewhat common pitfall in C is forgetting to implement a forward declaration,\nso banning them altogether is a valid move (like all other modern languages).",[13,237,238],{},"But without forward declaration, how do we handle dependency cycles?\nRight, we can allow dependency cycles in one (CMake-style C) \"library\"\nby considering all symbols declared within it at once.\nThe compile unit gets a little larger than the original C structure,\nbut it's still within control.",[13,240,241],{},"From here, to avoid the other common pitfall in C -- name clashing,\nwe make each \"library\" here its own namespace,\nso symbols within it won't clash with those outside.",[13,243,244],{},"And bam! We get a Go package.",[60,246,247],{":avatar":140},[13,248,249],{},"So, a Go package is basically a C library isolated in its own namespace,\nwith forward declaration replaced with cyclic references within the library,\nand compiled in one unit.",[13,251,252],{},"Doing so comes with costs. We'll see how Go's design deals with it.",[224,254,256],{"id":255},"interfaces","Interfaces",[229,258,259],{},[13,260,213],{},[13,262,263],{},"So, although we can remove cyclic dependencies between Go packages,\nthere's one more pattern we need to handle\nif we want people to write applications on it -- interfaces.",[13,265,266,267,270],{},"It's a ",[73,268,269],{},"very common"," pattern to declare interfaces and methods that use implementations together.\nTake this for an example:",[272,273,278],"pre",{"className":274,"code":276,"language":277},[275],"language-text","package pkg.services:\n  interface IService\n  class Factory:\n    method createA() uses ServiceA\n    method createB() uses ServiceB\n\npackage pkg.services.a:\n  class ServiceA impl IService\n\npackage pkg.services.b:\n  class ServiceB impl IService\n","text",[39,279,276],{"__ignoreMap":280},"",[13,282,283],{},"Such pattern is very common in applications --\ndefining high-level interfaces in one namespaces,\ndefining the implementations in the descendants of it.",[13,285,286,287,290,291,294,295,294,298,117],{},"But sadly, if you're still using nominal interfaces\n",[73,288,289],{},"while"," having everything else behave like Go packages,\nthis won't work.\nYou get a cyclic dependency:\n",[39,292,293],{},"pkg.services.Factory"," -> ",[39,296,297],{},"pkg.services.a.ServiceA",[39,299,300],{},"pkg.services.IService",[60,302,303],{":avatar":140},[13,304,305,306,309],{},"I can put ",[39,307,308],{},"IService"," into a separate Go package,\nand let all other packages depend on that, I guess?",[60,311,312],{":avatar":62,":flip":63},[13,313,314],{},"Yeah, why not.",[60,316,317],{":flip":63},[13,318,319,320,323,324,327],{},"Now you have to create ",[73,321,322],{},"a new package","\nfor each common dependency of\n",[73,325,326],{},"two packages"," you don't want to merge.\nEnjoy your packages!",[60,329,330],{":avatar":140},[13,331,332,337],{},[73,333,334],{},[73,335,336],{},"staring at dependency graph","\nOh man.",[13,339,340,341,344],{},"So technically you ",[73,342,343],{},"can"," solve it,\nbut at the cost of creating a lot more Go packages than you want.\nIf each file were their own \"package\" (like OCaml or TypeScript),\nyou might be able to pull that off (although still quite dirty),\nbut now you're working with folders,\nwhich are mentally heavier than files...",[13,346,347,348,351],{},"Go developers obviously went to the other route:\nWhat if our implementation ",[73,349,350],{},"does not"," depend on the interface?",[13,353,354,355,358,359,294,362,117],{},"This takes us to the land of structural interfaces.\nThe implementation ",[39,356,357],{},"ServiceA impl IService"," is now implicit and hidden to the compiler.\nSuch relationship checked only when you actually perform the cast on the type.\nIn this way, we can now remove the dependency ",[39,360,361],{},"pkg.services.a",[39,363,364],{},"pkg.services",[13,366,367],{},"So, the original downside of Go's package design is now somewhat fixed using structural interfaces,\nand structural interfaces (not nominal) are justified by the package design.\nReasoning cycle 1, complete.",[224,369,371],{"id":370},"go-generate",[39,372,218],{},[229,374,375],{},[13,376,377,219],{},[39,378,218],{},[13,380,381],{},"Macros. The part of C that programmer hate and love the most.\nThey are powerful enough to save programmers from writing repetitive code,\neven generating generic data structures at compile time.\nHowever (as what they are in C), they are notoriously hard to maintain.",[13,383,384],{},"This makes it a clear no-go for Go. (Haha, unintentional wordplay.)\nSo, nothing can modify existing source files now.",[60,386,387],{":avatar":140},[13,388,389],{},"Can't they use procedural macros?",[60,391,392],{":avatar":62,":flip":63},[13,393,394],{},"You know Go people will decline compile-time code execution ideas\nlike this at first glance.",[13,396,397,398,401],{},"But programmers still need ",[73,399,400],{},"something"," to generate code for them.\nThe thing will need to be able to read and parse existing code,\nand then inject definitions that interact with existing code.\nSince Go doesn't permit dependency cycles,\nthey will need to directly inject the code into the current Go package,\nbut without touching existing files.",[13,403,404,407],{},[73,405,406],{},"But wait!","\nIn Go packages, every file within a package already share the same namespace,\nso you can just put generated code into the current folder and call it a day!",[13,409,410,411,413,414,417,418,421],{},"This is what Go designers settled on.\nA ",[39,412,218],{}," command that invokes custom generator executables\n(that can parse and generate Go ASTs),\nwhich then generates needed code in ",[73,415,416],{},"new files","\nthat gets ",[73,419,420],{},"automagically included"," into the current package.",[60,423,424],{":avatar":140},[13,425,426,427,430],{},"But with this definition,\nthey ",[73,428,429],{},"are"," procedural macros,\njust manually invoked and generate into another file!",[60,432,433],{":avatar":62,":flip":63},[13,434,435],{},"Then every code generator is a procedural macro.",[60,437,438],{":flip":63},[13,439,440,445],{},[73,441,442],{},[73,443,444],{},"inhales","\nOh wait they are.",[13,447,448],{},"So in short,\nthe Go package design makes generating code in other files painless,\nand such code generation fixes the issue of not having macros to modify files,\nmaking Go's package design necessary.\nReasoning cycle 2, complete.",[224,450,452],{"id":451},"import-list","Import list",[229,454,455],{},[13,456,457],{},"Each file gets to specify its list of imported namespace.",[13,459,460],{},"Nobody likes managing dependency,\nincreasingly so when the unit of dependency gets smaller and smaller.",[13,462,463,464,467],{},"You might be able to manage the dependency list of a NPM ",[39,465,466],{},"package.json"," with ease.\nBut managing dependencies of libraries in a large project\nwhere some of its code are even generated?\nThat's another story.",[13,469,470,471,474],{},"Not only will you face a dependency graph\nmuch more detailed than the package-(not Go packages)-level,\nbut in generated code you ",[73,472,473],{},"will not know what namespaces the generated files depend on",",\nuntil the compiler complains!\n-- And it might change from time to time!",[13,476,477],{},"This is actually quite common in C\u002FC++ project management --\ndependencies are hard --\nand it's why many C\u002FC++ libraries are dependency-free and\u002For header-only.",[60,479,480],{":avatar":140},[13,481,482,483,486],{},"Since we usually explicitly import namespaces in code,\nand namespaces ",[73,484,485],{},"are exactly"," packages in Go,\ndoes that mean...",[13,488,489,490,492],{},"Go takes a clever way to solve that.\nInstead of the C way of having a ",[39,491,200],{},"-like file\nin each Go package specifying its dependencies,\nit just uses an import statement like any other modern languages,\nand calculates dependencies from those statements.",[13,494,495,496,498],{},"Because each namespace in Go directly corresponds to a Go package,\nthe union of all import statements ",[73,497,429],{}," the dependency of the package.\nThis also solves the problem of not knowing the dependencies of generated code --\nit's specified in that file, so you don't need to even think about managing them.",[13,500,501],{},"So, the design of code generation and Go packages can be justified\nby gathering dependencies directly from import lists,\nand using import list as dependency list can only work when packages is equal to namespaces.\nReasoning cycle 3, complete.",[224,503,505],{"id":504},"what-do-we-get-now","What do we get now?",[13,507,508],{},"As can be seen in this dependency graph of features in Go,\nthese features are clearly tightly coupled.\nIn its heart, the feature \"a namespace is exactly one compile unit\"\nhas its downside solved by other features,\nand justifies the existence of them.",[13,510,511],{},[512,513],"img",{"alt":514,"src":515},"Dependencies of Go packages and features","\u002Fposts\u002Fcompile-unit-neq-namespace\u002Fdependencies.svg",[60,517,518],{":avatar":140},[13,519,520],{},"Wondering if I can change some of them...",[165,522,524],{"id":523},"when-gos-solution-doesnt-work","When Go's solution doesn't work",[13,526,527],{},"Say you're designing something of your own.\nYour personal weekend programming language project or something like that.\nCan you use Go's solution on the trinity of namespaces, compile units, folder?",[13,529,530,531,534],{},"As you might have guessed from the graph above --\n",[24,532,533],{},"No. Absolutely no, unless you're making some carbon copy of Go itself.","\nEven then, unless you're absolutely sure it will work in your scenario, just don't.",[13,536,537],{},"Let's try doing an experiment.\nWhat if we change some of the features above?",[13,539,540],{},"Beginning with the structural interface feature.\nIf we replace that with nominal interfaces\n(which means you need to explicitly implement them), what will happen?",[60,542,543],{":avatar":140},[13,544,545],{},"Umm, I will create a new namespace (i.e. \"Go\" package) for the interface type,\nso that it can be imported by both its implementors and users?",[60,547,548],{},[13,549,550],{},"And because each namespace spans a whole folder,\nI will end up with many folders with only one file in each!",[13,552,553,554,557],{},"Might not be ",[73,555,556],{},"that"," disastrous as you might think,\nbut many small folders would still be pretty annoying to work with.\nIn that case you might better off just use file-scoped namespaces,\nlarger compile units or forward declaration.",[13,559,560,561,563],{},"Okay. Next up, ",[39,562,218],{},".\nWhat if we remove it?",[60,565,566],{":avatar":140},[13,567,568],{},"I will lose the ability to procedurally generate code.",[60,570,571],{},[13,572,573,574,117],{},"Okay maybe not,\nbut the ecosystem will definitely become more divided.\nEverybody will have their own version of ",[39,575,218],{},[13,577,578],{},"Right, metaprogramming is required for compiled languages like this,\nor people will find the 100 other workarounds to do metaprogramming without official support for it.",[13,580,581,582,585,586,589],{},"On the other side, if you think of replacing it with ",[73,583,584],{},"real"," procedural macros...\nThat's a pure boost,\nbut it also means you won't need the generated code\nto be ",[73,587,588],{},"automagically"," included in the same namespace as your source code,\nso the namespace design is useless again.",[13,591,592],{},"Finally, what if we remove the import list?",[60,594,595],{":avatar":140},[13,596,597],{},[73,598,599,600,602],{},"I DON'T WANT TO WRITE ",[39,601,200],{}," AGAIN!!!!!!",[13,604,605],{},"That tells.\nMicromanaging dependencies is never a good idea,\nso in this way you'll prefer the two mainstream designs again.",[13,607,608,609,612,613,616],{},"And let's not forget the elephant in the room -- ",[24,610,611],{},"large dependency cycles",".\nIf you happen to be writing something that has a large inherent complexity,\nlike compilers or large applications,\nyou'll probably encounter one or two very large dependency cycles\nthat ",[73,614,615],{},"just can't"," be easily reduced without increasing maintenance difficulty.",[13,618,619,620,623,624,627,628,631],{},"With Go's namespace design,\nthe tool you have to manage complexity --\nsmaller namespaces that can be cyclically referenced and partially exported --\ndoes not exist ",[73,621,622],{},"at all",".\nYou're forced to put ",[73,625,626],{},"everything"," in your reference cycle\ninto ",[24,629,630],{},"one big f_cking flat namespace",".\nIt'd be nightmare maintaining code like this.",[165,633,635],{"id":634},"conclusion","Conclusion",[60,637,638],{":avatar":62,":flip":63},[13,639,640,641],{},"Do I need to state this ",[73,642,643],{},"again?!",[13,645,646],{},[24,647,648,649,652],{},"Go's solution works, but ",[73,650,651],{},"it's because it's Go",". Go is special.",[13,654,655],{},"Go has its very specific set of trade-offs it made,\nso that making compile units precisely equal to namespace is possible.",[13,657,658],{},[24,659,660],{},"If you're designing something on your own,\nyou're very likely not using the same set of trade-offs as Go does.\nSo, just. Please. DO NOT COPY GO'S NAMESPACE DESIGN.",[13,662,663],{},"Take a walk outside. Get some fresh air. Touch the grass.\nUse anything sane from scripting language's per-file scope,\nto modern languages' package-level compile units.\nCaching will always be your friend.\nEven C\u002FC++'s forward declaration and separated compilation has some merits.\nBut you're not Go, so just don't copy Go.",{"title":280,"searchDepth":665,"depth":665,"links":666},2,[667,675,676],{"id":167,"depth":665,"text":168,"children":668},[669,671,672,673,674],{"id":226,"depth":670,"text":227},3,{"id":255,"depth":670,"text":256},{"id":370,"depth":670,"text":218},{"id":451,"depth":670,"text":452},{"id":504,"depth":670,"text":505},{"id":523,"depth":665,"text":524},{"id":634,"depth":665,"text":635},"2025-03-15","A brief research about the relationship between compile unit and namespaces, and why you shouldn't copy Go on that.\n","md",{},true,"\u002Fposts\u002Fcompile-unit-neq-namespace",{"title":5,"description":678},"posts\u002Fcompile-unit-neq-namespace\u002Findex","HDI-glXjHFCRF4MaccD0NDqmL2cG-hI-JWq_8viwU4E",1786371125292]